2024-09-16T16:32:34.177 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T16:32:34.193 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T16:32:34.269 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7907389 branch: wip-matanb-crimson-testing-59543 description: crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore tasks/rbd_api_tests} email: null first_in_suite: false job_id: '7907389' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-matanb-crimson-testing-59543 ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mgr stats period: 30 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson alien thread cpu cores: 6-7 crimson osd obc lru size: 10 debug ms: 1 debug osd: 20 osd objectstore: seastore osd shutdown pgref assert: true osd.0: crimson seastar cpu cores: 0-1 osd.1: crimson seastar cpu cores: 2-3 osd.2: crimson seastar cpu cores: 4-5 flavor: crimson fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_APP_NOT_ENABLED\) - is full \(reached quota - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(POOL_FULL\) sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: crimson sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 owner: scheduled_matan@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6678 sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 0325fd3c11a225c0fd9eb38fab88aad1370ad779 targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCoOyiajz1larOvU1hiCsPOS5mx8iADdKwpvxxMg3tEZEscWuBWp5PL+KdhS1za8sPAgi3eHFtdRCfs6HFKWImA= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rbd/crimson/test_crimson_librbd.sh env: RBD_FEATURES: '61' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_16:23:48 tube: smithi user: matan verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T16:32:34.269 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa; will attempt to use it 2024-09-16T16:32:34.270 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa/tasks 2024-09-16T16:32:34.270 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T16:32:34.271 INFO:teuthology.task.internal:Checking packages... 2024-09-16T16:32:34.293 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '81fedcb3b484c843d22ffbc8d179cb567b7083b5' 2024-09-16T16:32:34.293 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T16:32:34.293 INFO:teuthology.packaging:ref: None 2024-09-16T16:32:34.293 INFO:teuthology.packaging:tag: None 2024-09-16T16:32:34.293 INFO:teuthology.packaging:branch: wip-matanb-crimson-testing-59543 2024-09-16T16:32:34.293 INFO:teuthology.packaging:sha1: 81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-16T16:32:34.293 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&ref=wip-matanb-crimson-testing-59543 2024-09-16T16:32:34.432 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4879.g81fedcb3 2024-09-16T16:32:34.434 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T16:32:34.440 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T16:32:34.440 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T16:32:34.465 INFO:teuthology.task.internal:Saving configuration 2024-09-16T16:32:34.477 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T16:32:34.490 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T16:32:34.510 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7907389', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 16:24:58.084755', 'locked_by': 'scheduled_matan@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCoOyiajz1larOvU1hiCsPOS5mx8iADdKwpvxxMg3tEZEscWuBWp5PL+KdhS1za8sPAgi3eHFtdRCfs6HFKWImA='} 2024-09-16T16:32:34.511 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T16:32:34.548 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2024-09-16T16:32:34.549 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T16:32:34.595 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f23bd28bbe0>, signals=[15]) 2024-09-16T16:32:34.595 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T16:32:34.605 INFO:teuthology.task.internal:Opening connections... 2024-09-16T16:32:34.606 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T16:32:34.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T16:32:34.684 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T16:32:34.690 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-09-16T16:32:34.763 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-09-16T16:32:34.763 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-09-16T16:32:34.821 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-09-16T16:32:34.822 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T16:32:34.822 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-09-16T16:32:34.822 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T16:32:34.822 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T16:32:34.822 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T16:32:34.822 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-09-16T16:32:34.839 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T16:32:34.848 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T16:32:34.882 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T16:32:34.882 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-09-16T16:32:34.900 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T16:32:34.914 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T16:32:34.914 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-09-16T16:32:35.069 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T16:32:35.097 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T16:32:35.111 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-09-16T16:32:35.111 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-09-16T16:32:35.111 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T16:32:35.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T16:32:35.129 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-09-16T16:32:35.187 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-503.el9.x86_64 2024-09-16T16:32:35.187 INFO:teuthology.task.kernel:Running kernel on smithi123: 5.14.0-503.el9.x86_64 2024-09-16T16:32:35.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2024-09-16T16:32:37.424 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.2 MB 00:00 2024-09-16T16:32:39.353 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-16T16:32:44.301 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 5.6 MB/s | 6.5 MB 00:01 2024-09-16T16:32:45.984 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 19 kB 00:00 2024-09-16T16:32:47.139 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-16T16:32:52.494 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T16:32:54.567 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T16:32:54.567 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T16:32:54.644 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:32:54.649 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:32:54.650 INFO:teuthology.orchestra.run.smithi123.stdout:Install 4 Packages 2024-09-16T16:32:54.651 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:32:54.651 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 79 M 2024-09-16T16:32:54.651 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 126 M 2024-09-16T16:32:54.651 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:32:54.987 INFO:teuthology.orchestra.run.smithi123.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 278 kB/s | 23 kB 00:00 2024-09-16T16:32:55.596 INFO:teuthology.orchestra.run.smithi123.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-16T16:32:56.388 INFO:teuthology.orchestra.run.smithi123.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 23 MB/s | 35 MB 00:01 2024-09-16T16:32:57.121 INFO:teuthology.orchestra.run.smithi123.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-16T16:32:57.122 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:32:57.122 INFO:teuthology.orchestra.run.smithi123.stdout:Total 32 MB/s | 79 MB 00:02 2024-09-16T16:32:57.796 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:32:58.152 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:32:58.152 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:32:59.091 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:32:59.091 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:33:01.532 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:33:02.182 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T16:33:02.263 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T16:33:05.063 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T16:33:05.199 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T16:33:10.067 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T16:33:10.191 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T16:33:15.004 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T16:33:52.933 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T16:33:52.972 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T16:33:53.419 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T16:33:53.419 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T16:33:53.419 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T16:33:53.419 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T16:33:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T16:33:54.262 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:33:54.262 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:33:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T16:33:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T16:33:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T16:33:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T16:33:54.263 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:33:54.263 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:33:54.627 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T16:33:54.714 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T16:33:54.714 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T16:33:54.714 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T16:33:54.714 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T16:33:54.715 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T16:33:54.715 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T16:33:54.715 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T16:33:54.715 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T16:33:54.774 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T16:33:54.774 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T16:33:54.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-09-16T16:33:54.809 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-09-16T16:33:54.809 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-09-16T16:33:54.809 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-09-16T16:33:54.809 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 82.el9 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8274670 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T16:33:54.810 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:33:54.811 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T16:33:54.812 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T16:33:54.812 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 5.14.0-505.el9.x86_64 2024-09-16T16:33:54.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T16:33:55.430 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-09-16T16:33:56.774 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T16:33:56.809 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-09-16T16:33:56.811 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-09-16T16:33:56.826 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.1CZfTkyRE5 2024-09-16T16:33:56.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1CZfTkyRE5 2024-09-16T16:33:56.898 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.1CZfTkyRE5 2024-09-16T16:33:57.031 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.1CZfTkyRE5 is 6KB 2024-09-16T16:33:57.079 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.1CZfTkyRE5 2024-09-16T16:33:57.094 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T16:33:57.157 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T16:33:57.157 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T16:33:57.157 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T16:33:57.157 INFO:teuthology.orchestra.run.smithi123.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T16:33:57.158 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T16:33:57.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-09-16T16:34:27.340 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T16:34:27.341 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-16T16:34:27.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T16:35:01.822 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-09-16T16:35:10.829 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-16T16:35:10.831 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T16:35:29.214 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2024-09-16T16:35:41.226 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-16T16:35:41.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T16:35:42.155 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-16T16:35:42.591 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-16T16:35:42.592 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T16:35:42.592 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-09-16T16:35:42.606 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-505.el9.x86_64 2024-09-16T16:35:42.607 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T16:35:42.607 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T16:35:42.607 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T16:35:43.607 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T16:35:43.608 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T16:35:43.693 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-09-16T16:35:43.723 DEBUG:teuthology.parallel:result is None 2024-09-16T16:35:43.723 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T16:35:43.730 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T16:35:43.731 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T16:35:43.746 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T16:35:43.753 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T16:35:43.785 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T16:35:43.785 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T16:35:43.811 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T16:35:43.842 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T16:35:43.842 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T16:35:43.873 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T16:35:43.883 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T16:35:43.885 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T16:35:43.891 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T16:35:43.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T16:35:43.948 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T16:35:43.957 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T16:35:43.958 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T16:35:44.001 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T16:35:44.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T16:35:44.123 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T16:35:44.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T16:35:44.241 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:35:44.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T16:35:44.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-09-16T16:35:44.394 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T16:35:44.615 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T16:35:44.626 INFO:teuthology.task.internal:Starting timer... 2024-09-16T16:35:44.626 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T16:35:44.675 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T16:35:44.714 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-16T16:35:44.714 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-09-16T16:35:44.767 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-09-16T16:35:44.768 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T16:35:44.769 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-09-16T16:35:44.793 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-09-16T16:35:44.793 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2024-09-16T16:35:44.794 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T16:35:44.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T16:35:44.855 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T16:35:44.855 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-09-16T16:35:44.921 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T16:35:44.931 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-16T16:35:44.933 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-16T16:35:45.046 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T16:35:45.058 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T16:35:45.059 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T16:40:05.326 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-09-16T16:40:05.327 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-16T16:40:05.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T16:40:05.416 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-16T16:40:05.500 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-16T16:40:05.500 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T16:40:05.511 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T16:40:05.513 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T16:40:05.514 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T16:40:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T16:40:05.582 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T16:40:05.607 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-09-16T16:40:05.618 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-09-16T16:40:05.632 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T16:40:05.646 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T16:40:05.699 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-16T16:40:05.704 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T16:40:05.705 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-16T16:40:05.705 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T16:40:05.705 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T16:40:05.705 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T16:40:05.705 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T16:40:05.705 INFO:teuthology.run_tasks:Running task install... 2024-09-16T16:40:05.713 DEBUG:teuthology.task.install:project ceph 2024-09-16T16:40:05.713 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}} 2024-09-16T16:40:05.713 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'} 2024-09-16T16:40:05.714 INFO:teuthology.task.install:Using flavor: crimson 2024-09-16T16:40:05.725 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T16:40:05.725 INFO:teuthology.task.install:extra packages: [] 2024-09-16T16:40:05.726 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5', 'tag': None, 'wait_for_package': False} 2024-09-16T16:40:05.727 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-16T16:40:05.875 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/ 2024-09-16T16:40:05.875 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4879.g81fedcb3 2024-09-16T16:40:06.000 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-matanb-crimson-testing-59543/81fedcb3b484c843d22ffbc8d179cb567b7083b5/centos/9/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T16:40:06.001 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:40:06.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T16:40:06.028 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-16T16:40:06.028 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81fedcb3b484c843d22ffbc8d179cb567b7083b5/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T16:40:06.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T16:40:06.169 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T16:40:06.233 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-09-16T16:40:06.234 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-09-16T16:40:06.466 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-09-16T16:40:06.492 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw 2024-09-16T16:40:07.410 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 140 kB/s | 88 kB 00:00 2024-09-16T16:40:07.902 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 53 kB/s | 24 kB 00:00 2024-09-16T16:40:08.353 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 5.0 kB/s | 2.1 kB 00:00 2024-09-16T16:40:11.038 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-09-16T16:40:23.836 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 1.7 MB/s | 20 MB 00:11 2024-09-16T16:40:28.461 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.5 MB 00:00 2024-09-16T16:40:30.437 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 19 kB 00:00 2024-09-16T16:40:32.008 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-16T16:40:37.372 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-16T16:40:39.715 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 10 M 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.5 M 2024-09-16T16:40:39.733 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.8 M 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 6.1 M 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 20 M 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 25 k 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 809 k 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T16:40:39.734 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 558 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 5.4 M 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 45 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 143 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 160 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 315 k 2024-09-16T16:40:39.735 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 302 k 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 99 k 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout:Install 31 Packages 2024-09-16T16:40:39.736 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 2 Packages 2024-09-16T16:40:39.737 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:40:39.737 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 60 M 2024-09-16T16:40:39.737 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:40:46.922 INFO:teuthology.orchestra.run.smithi123.stdout:(1/33): ceph-base-19.3.0-4879.g81fedcb3.el9.x86 886 kB/s | 6.1 MB 00:07 2024-09-16T16:40:47.022 INFO:teuthology.orchestra.run.smithi123.stdout:(2/33): ceph-selinux-19.3.0-4879.g81fedcb3.el9. 251 kB/s | 25 kB 00:00 2024-09-16T16:40:47.930 INFO:teuthology.orchestra.run.smithi123.stdout:(3/33): libcephfs2-19.3.0-4879.g81fedcb3.el9.x8 891 kB/s | 809 kB 00:00 2024-09-16T16:40:48.489 INFO:teuthology.orchestra.run.smithi123.stdout:(4/33): libradosstriper1-19.3.0-4879.g81fedcb3. 1.0 MB/s | 558 kB 00:00 2024-09-16T16:40:52.639 INFO:teuthology.orchestra.run.smithi123.stdout:(5/33): librgw2-19.3.0-4879.g81fedcb3.el9.x86_6 1.3 MB/s | 5.4 MB 00:04 2024-09-16T16:40:52.747 INFO:teuthology.orchestra.run.smithi123.stdout:(6/33): python3-ceph-argparse-19.3.0-4879.g81fe 417 kB/s | 45 kB 00:00 2024-09-16T16:40:52.989 INFO:teuthology.orchestra.run.smithi123.stdout:(7/33): python3-ceph-common-19.3.0-4879.g81fedc 592 kB/s | 143 kB 00:00 2024-09-16T16:40:53.239 INFO:teuthology.orchestra.run.smithi123.stdout:(8/33): python3-cephfs-19.3.0-4879.g81fedcb3.el 638 kB/s | 160 kB 00:00 2024-09-16T16:40:53.648 INFO:teuthology.orchestra.run.smithi123.stdout:(9/33): python3-rados-19.3.0-4879.g81fedcb3.el9 772 kB/s | 315 kB 00:00 2024-09-16T16:40:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:(10/33): python3-rbd-19.3.0-4879.g81fedcb3.el9. 614 kB/s | 302 kB 00:00 2024-09-16T16:40:54.540 INFO:teuthology.orchestra.run.smithi123.stdout:(11/33): python3-rgw-19.3.0-4879.g81fedcb3.el9. 249 kB/s | 99 kB 00:00 2024-09-16T16:40:54.699 INFO:teuthology.orchestra.run.smithi123.stdout:(12/33): boost-program-options-1.75.0-8.el9.x86 676 kB/s | 107 kB 00:00 2024-09-16T16:40:54.774 INFO:teuthology.orchestra.run.smithi123.stdout:(13/33): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-16T16:40:54.858 INFO:teuthology.orchestra.run.smithi123.stdout:(14/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-16T16:40:54.899 INFO:teuthology.orchestra.run.smithi123.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-16T16:40:55.100 INFO:teuthology.orchestra.run.smithi123.stdout:(16/33): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-09-16T16:40:55.208 INFO:teuthology.orchestra.run.smithi123.stdout:(17/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-16T16:40:55.250 INFO:teuthology.orchestra.run.smithi123.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-16T16:40:55.334 INFO:teuthology.orchestra.run.smithi123.stdout:(19/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-09-16T16:40:55.376 INFO:teuthology.orchestra.run.smithi123.stdout:(20/33): qatlib-service-24.02.0-1.el9.x86_64.rp 868 kB/s | 36 kB 00:00 2024-09-16T16:40:55.418 INFO:teuthology.orchestra.run.smithi123.stdout:(21/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-16T16:40:55.468 INFO:teuthology.orchestra.run.smithi123.stdout:(22/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.0 MB/s | 308 kB 00:00 2024-09-16T16:40:55.618 INFO:teuthology.orchestra.run.smithi123.stdout:(23/33): libarrow-9.0.0-11.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-09-16T16:40:55.644 INFO:teuthology.orchestra.run.smithi123.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T16:40:55.669 INFO:teuthology.orchestra.run.smithi123.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T16:40:55.694 INFO:teuthology.orchestra.run.smithi123.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T16:40:55.795 INFO:teuthology.orchestra.run.smithi123.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-16T16:40:55.903 INFO:teuthology.orchestra.run.smithi123.stdout:(28/33): ceph-radosgw-19.3.0-4879.g81fedcb3.el9 665 kB/s | 10 MB 00:15 2024-09-16T16:40:55.928 INFO:teuthology.orchestra.run.smithi123.stdout:(29/33): re2-20211101-20.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-16T16:40:56.087 INFO:teuthology.orchestra.run.smithi123.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-09-16T16:41:01.220 INFO:teuthology.orchestra.run.smithi123.stdout:(31/33): librados2-19.3.0-4879.g81fedcb3.el9.x8 672 kB/s | 3.5 MB 00:05 2024-09-16T16:41:01.679 INFO:teuthology.orchestra.run.smithi123.stdout:(32/33): ceph-common-19.3.0-4879.g81fedcb3.el9. 930 kB/s | 20 MB 00:21 2024-09-16T16:41:03.087 INFO:teuthology.orchestra.run.smithi123.stdout:(33/33): librbd1-19.3.0-4879.g81fedcb3.el9.x86_ 555 kB/s | 3.8 MB 00:06 2024-09-16T16:41:03.092 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:41:03.092 INFO:teuthology.orchestra.run.smithi123.stdout:Total 2.6 MB/s | 60 MB 00:23 2024-09-16T16:41:03.201 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:41:03.238 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:41:03.238 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:41:03.903 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:41:03.904 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:41:05.400 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:41:05.922 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-16T16:41:06.014 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-16T16:41:06.106 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-16T16:41:06.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/35 2024-09-16T16:41:06.232 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-16T16:41:06.296 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-16T16:41:06.350 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 5/35 2024-09-16T16:41:06.379 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/35 2024-09-16T16:41:06.412 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/35 2024-09-16T16:41:06.435 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-16T16:41:06.453 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 8/35 2024-09-16T16:41:06.622 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-16T16:41:06.659 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 9/35 2024-09-16T16:41:06.694 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/35 2024-09-16T16:41:06.738 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9. 11/35 2024-09-16T16:41:06.767 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 12/35 2024-09-16T16:41:06.845 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 13/35 2024-09-16T16:41:06.872 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 14/35 2024-09-16T16:41:06.958 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 14/35 2024-09-16T16:41:06.987 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 15/35 2024-09-16T16:41:07.095 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/35 2024-09-16T16:41:07.129 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 17/35 2024-09-16T16:41:07.172 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 18/35 2024-09-16T16:41:07.220 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 19/35 2024-09-16T16:41:07.249 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 20/35 2024-09-16T16:41:07.288 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 21/35 2024-09-16T16:41:07.653 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-16T16:41:07.704 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 23/35 2024-09-16T16:41:07.794 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 23/35 2024-09-16T16:41:07.834 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 24/35 2024-09-16T16:41:07.877 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-16T16:41:08.454 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-16T16:41:08.481 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 27/35 2024-09-16T16:41:08.553 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 27/35 2024-09-16T16:41:08.647 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 28/35 2024-09-16T16:41:08.683 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x8 29/35 2024-09-16T16:41:11.125 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-16T16:41:11.171 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-16T16:41:11.905 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 30/35 2024-09-16T16:41:11.957 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 31/35 2024-09-16T16:41:12.048 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 31/35 2024-09-16T16:41:12.049 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T16:41:12.049 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T16:41:12.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:41:12.075 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-16T16:41:21.992 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-16T16:41:21.992 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-09-16T16:41:21.992 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-09-16T16:41:21.992 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-09-16T16:41:21.992 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-09-16T16:41:21.992 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-09-16T16:41:21.992 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-09-16T16:41:21.993 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-09-16T16:41:21.993 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:41:23.107 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/35 2024-09-16T16:41:23.132 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/35 2024-09-16T16:41:23.132 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:41:23.132 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T16:41:23.132 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T16:41:23.132 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T16:41:23.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:41:23.134 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T16:41:23.243 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-16T16:41:23.244 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 5/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_6 6/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 7/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9. 8/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x8 9/35 2024-09-16T16:41:24.623 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 10/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 11/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 12/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 13/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-16T16:41:24.624 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-16T16:41:24.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-16T16:41:24.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-16T16:41:24.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-16T16:41:24.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-16T16:41:24.625 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-16T16:41:24.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-16T16:41:24.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-16T16:41:24.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-16T16:41:24.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-16T16:41:24.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 32/35 2024-09-16T16:41:24.626 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-16T16:41:24.627 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/35 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.550 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.551 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T16:41:25.552 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T16:41:25.553 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:41:25.553 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:41:25.884 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-test 2024-09-16T16:41:26.450 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:00:49 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:41:26.594 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:41:26.595 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:41:26.595 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 85 M 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: libasan8 x86_64 14.2.1-1.2.el9 appstream 513 k 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 175 k 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: libubsan x86_64 11.5.0-2.el9 appstream 183 k 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2024-09-16T16:41:26.596 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:41:26.597 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:41:26.597 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:41:26.597 INFO:teuthology.orchestra.run.smithi123.stdout:Install 9 Packages 2024-09-16T16:41:26.597 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:41:26.597 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 87 M 2024-09-16T16:41:26.597 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 797 M 2024-09-16T16:41:26.598 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:41:27.023 INFO:teuthology.orchestra.run.smithi123.stdout:(1/9): lksctp-tools-1.0.19-2.el9.x86_64.rpm 534 kB/s | 94 kB 00:00 2024-09-16T16:41:27.173 INFO:teuthology.orchestra.run.smithi123.stdout:(2/9): libasan8-14.2.1-1.2.el9.x86_64.rpm 3.4 MB/s | 513 kB 00:00 2024-09-16T16:41:27.215 INFO:teuthology.orchestra.run.smithi123.stdout:(3/9): libubsan-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 183 kB 00:00 2024-09-16T16:41:27.432 INFO:teuthology.orchestra.run.smithi123.stdout:(4/9): protobuf-3.14.0-13.el9.x86_64.rpm 4.7 MB/s | 1.0 MB 00:00 2024-09-16T16:41:27.499 INFO:teuthology.orchestra.run.smithi123.stdout:(5/9): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-16T16:41:27.549 INFO:teuthology.orchestra.run.smithi123.stdout:(6/9): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T16:41:27.591 INFO:teuthology.orchestra.run.smithi123.stdout:(7/9): yaml-cpp-0.6.3-5.el9.x86_64.rpm 2.9 MB/s | 122 kB 00:00 2024-09-16T16:41:28.166 INFO:teuthology.orchestra.run.smithi123.stdout:(8/9): libcephsqlite-19.3.0-4879.g81fedcb3.el9. 133 kB/s | 175 kB 00:01 2024-09-16T16:42:41.132 INFO:teuthology.orchestra.run.smithi123.stdout:(9/9): ceph-test-19.3.0-4879.g81fedcb3.el9.x86_ 1.1 MB/s | 85 MB 01:14 2024-09-16T16:42:41.134 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:42:41.134 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.2 MB/s | 87 MB 01:14 2024-09-16T16:42:41.205 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:42:41.226 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:42:41.226 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:42:41.866 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:42:41.866 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:42:42.748 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:42:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/9 2024-09-16T16:42:42.884 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 2/9 2024-09-16T16:42:42.935 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 3/9 2024-09-16T16:42:42.962 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 4/9 2024-09-16T16:42:42.993 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-16T16:42:43.018 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-16T16:42:43.093 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libasan8-14.2.1-1.2.el9.x86_64 6/9 2024-09-16T16:42:43.136 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 7/9 2024-09-16T16:42:43.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 8/9 2024-09-16T16:42:54.335 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 8/9 2024-09-16T16:42:54.418 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 9/9 2024-09-16T16:43:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 9/9 2024-09-16T16:43:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/9 2024-09-16T16:43:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/9 2024-09-16T16:43:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 3/9 2024-09-16T16:43:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libasan8-14.2.1-1.2.el9.x86_64 4/9 2024-09-16T16:43:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libubsan-11.5.0-2.el9.x86_64 5/9 2024-09-16T16:43:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 6/9 2024-09-16T16:43:00.638 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 7/9 2024-09-16T16:43:00.638 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 8/9 2024-09-16T16:43:01.055 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 9/9 2024-09-16T16:43:01.055 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:43:01.055 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:43:01.055 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:43:01.055 INFO:teuthology.orchestra.run.smithi123.stdout: libasan8-14.2.1-1.2.el9.x86_64 2024-09-16T16:43:01.055 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout: libubsan-11.5.0-2.el9.x86_64 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:43:01.056 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:43:01.388 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph 2024-09-16T16:43:01.955 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:02:24 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:43:02.091 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 6.5 k 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 2.2 M 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 1.6 M 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 291 k 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 5.1 M 2024-09-16T16:43:02.094 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 70 M 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T16:43:02.095 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T16:43:02.096 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout:Install 37 Packages 2024-09-16T16:43:02.097 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:43:02.098 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 84 M 2024-09-16T16:43:02.098 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 897 M 2024-09-16T16:43:02.098 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:43:03.271 INFO:teuthology.orchestra.run.smithi123.stdout:(1/37): ceph-19.3.0-4879.g81fedcb3.el9.x86_64.r 18 kB/s | 6.5 kB 00:00 2024-09-16T16:43:04.980 INFO:teuthology.orchestra.run.smithi123.stdout:(2/37): ceph-mds-19.3.0-4879.g81fedcb3.el9.x86_ 1.1 MB/s | 2.2 MB 00:02 2024-09-16T16:43:06.496 INFO:teuthology.orchestra.run.smithi123.stdout:(3/37): ceph-mgr-19.3.0-4879.g81fedcb3.el9.x86_ 455 kB/s | 1.6 MB 00:03 2024-09-16T16:43:07.063 INFO:teuthology.orchestra.run.smithi123.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4879.g81fe 513 kB/s | 291 kB 00:00 2024-09-16T16:43:07.255 INFO:teuthology.orchestra.run.smithi123.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-16T16:43:07.364 INFO:teuthology.orchestra.run.smithi123.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 11 MB/s | 1.2 MB 00:00 2024-09-16T16:43:07.423 INFO:teuthology.orchestra.run.smithi123.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-16T16:43:07.473 INFO:teuthology.orchestra.run.smithi123.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.7 MB/s | 135 kB 00:00 2024-09-16T16:43:07.523 INFO:teuthology.orchestra.run.smithi123.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-16T16:43:07.573 INFO:teuthology.orchestra.run.smithi123.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-16T16:43:07.749 INFO:teuthology.orchestra.run.smithi123.stdout:(11/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-16T16:43:07.832 INFO:teuthology.orchestra.run.smithi123.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-16T16:43:07.916 INFO:teuthology.orchestra.run.smithi123.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 418 kB/s | 35 kB 00:00 2024-09-16T16:43:07.999 INFO:teuthology.orchestra.run.smithi123.stdout:(14/37): python3-packaging-20.9-5.el9.noarch.rp 926 kB/s | 77 kB 00:00 2024-09-16T16:43:08.074 INFO:teuthology.orchestra.run.smithi123.stdout:(15/37): python3-pytz-2021.1-5.el9.noarch.rpm 680 kB/s | 51 kB 00:00 2024-09-16T16:43:08.175 INFO:teuthology.orchestra.run.smithi123.stdout:(16/37): python3-toml-0.10.2-6.el9.noarch.rpm 417 kB/s | 42 kB 00:00 2024-09-16T16:43:08.283 INFO:teuthology.orchestra.run.smithi123.stdout:(17/37): lua-devel-5.4.4-4.el9.x86_64.rpm 206 kB/s | 22 kB 00:00 2024-09-16T16:43:08.334 INFO:teuthology.orchestra.run.smithi123.stdout:(18/37): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-16T16:43:08.359 INFO:teuthology.orchestra.run.smithi123.stdout:(19/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2024-09-16T16:43:08.393 INFO:teuthology.orchestra.run.smithi123.stdout:(20/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 5.0 MB/s | 172 kB 00:00 2024-09-16T16:43:08.443 INFO:teuthology.orchestra.run.smithi123.stdout:(21/37): python3-cherrypy-18.6.1-2.el9.noarch.r 7.0 MB/s | 358 kB 00:00 2024-09-16T16:43:08.468 INFO:teuthology.orchestra.run.smithi123.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 431 kB/s | 11 kB 00:00 2024-09-16T16:43:08.510 INFO:teuthology.orchestra.run.smithi123.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 424 kB/s | 18 kB 00:00 2024-09-16T16:43:08.535 INFO:teuthology.orchestra.run.smithi123.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 933 kB/s | 23 kB 00:00 2024-09-16T16:43:08.561 INFO:teuthology.orchestra.run.smithi123.stdout:(25/37): python3-jaraco-functools-3.5.0-2.el9.n 782 kB/s | 19 kB 00:00 2024-09-16T16:43:08.586 INFO:teuthology.orchestra.run.smithi123.stdout:(26/37): python3-jaraco-text-3.2.0-6.el9.noarch 795 kB/s | 20 kB 00:00 2024-09-16T16:43:08.611 INFO:teuthology.orchestra.run.smithi123.stdout:(27/37): python3-logutils-0.3.5-21.el9.noarch.r 1.8 MB/s | 46 kB 00:00 2024-09-16T16:43:08.637 INFO:teuthology.orchestra.run.smithi123.stdout:(28/37): python3-more-itertools-8.12.0-2.el9.no 3.1 MB/s | 79 kB 00:00 2024-09-16T16:43:08.670 INFO:teuthology.orchestra.run.smithi123.stdout:(29/37): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T16:43:08.695 INFO:teuthology.orchestra.run.smithi123.stdout:(30/37): python3-portend-3.1.0-2.el9.noarch.rpm 658 kB/s | 16 kB 00:00 2024-09-16T16:43:08.721 INFO:teuthology.orchestra.run.smithi123.stdout:(31/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.5 MB/s | 90 kB 00:00 2024-09-16T16:43:08.746 INFO:teuthology.orchestra.run.smithi123.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-16T16:43:08.780 INFO:teuthology.orchestra.run.smithi123.stdout:(33/37): python3-webob-1.8.8-2.el9.noarch.rpm 6.8 MB/s | 230 kB 00:00 2024-09-16T16:43:08.830 INFO:teuthology.orchestra.run.smithi123.stdout:(34/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 8.4 MB/s | 427 kB 00:00 2024-09-16T16:43:08.855 INFO:teuthology.orchestra.run.smithi123.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 806 kB/s | 20 kB 00:00 2024-09-16T16:43:11.572 INFO:teuthology.orchestra.run.smithi123.stdout:(36/37): ceph-mon-19.3.0-4879.g81fedcb3.el9.x86 630 kB/s | 5.1 MB 00:08 2024-09-16T16:43:59.873 INFO:teuthology.orchestra.run.smithi123.stdout:(37/37): ceph-osd-19.3.0-4879.g81fedcb3.el9.x86 1.3 MB/s | 70 MB 00:54 2024-09-16T16:43:59.876 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:43:59.877 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.5 MB/s | 84 MB 00:57 2024-09-16T16:44:00.020 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:44:00.061 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:44:00.061 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:44:00.861 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:44:00.861 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:44:02.518 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:44:02.575 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-16T16:44:02.617 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-16T16:44:02.644 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-16T16:44:02.723 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-16T16:44:02.768 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-16T16:44:02.801 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-16T16:44:02.893 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-16T16:44:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-16T16:44:03.020 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-16T16:44:03.055 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-16T16:44:03.093 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-16T16:44:03.136 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-16T16:44:03.169 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-16T16:44:03.210 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-16T16:44:03.245 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-16T16:44:03.279 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-16T16:44:03.327 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-16T16:44:03.360 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-16T16:44:03.433 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-16T16:44:03.572 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-16T16:44:03.668 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-16T16:44:03.721 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-16T16:44:03.759 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-16T16:44:03.808 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-16T16:44:03.864 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-16T16:44:04.051 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-16T16:44:04.104 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-16T16:44:04.174 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-16T16:44:04.274 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-16T16:44:04.335 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-16T16:44:04.469 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-16T16:44:04.659 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9. 32/37 2024-09-16T16:44:04.682 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/37 2024-09-16T16:44:04.709 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 33/37 2024-09-16T16:44:04.709 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:44:04.709 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T16:44:04.709 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T16:44:04.709 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T16:44:04.709 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:15.282 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/37 2024-09-16T16:44:15.310 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 34/37 2024-09-16T16:44:15.310 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:44:15.310 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T16:44:15.310 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T16:44:15.310 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T16:44:15.310 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 35/37 2024-09-16T16:44:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 35/37 2024-09-16T16:44:15.874 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:44:15.874 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T16:44:15.874 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T16:44:15.874 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T16:44:15.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:16.109 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 36/37 2024-09-16T16:44:16.132 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 36/37 2024-09-16T16:44:16.132 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:44:16.132 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T16:44:16.133 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T16:44:16.133 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T16:44:16.133 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:16.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 37/37 2024-09-16T16:44:22.666 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 37/37 2024-09-16T16:44:22.666 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/37 2024-09-16T16:44:22.666 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 3/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 4/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 5/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9. 6/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-16T16:44:22.667 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-16T16:44:22.668 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-16T16:44:22.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-16T16:44:22.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-16T16:44:22.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-16T16:44:22.669 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-16T16:44:22.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-16T16:44:22.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-16T16:44:22.670 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-16T16:44:23.612 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-16T16:44:23.612 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:23.612 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T16:44:23.613 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T16:44:23.614 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T16:44:23.615 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T16:44:23.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T16:44:23.616 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T16:44:23.616 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:23.616 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:44:23.921 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-base 2024-09-16T16:44:24.487 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:03:47 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:44:24.577 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-base-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:44:24.620 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:44:24.620 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:44:24.621 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:44:24.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install cephadm 2024-09-16T16:44:25.254 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:03:48 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:44:25.385 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 785 k 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:44:25.386 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:25.387 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 785 k 2024-09-16T16:44:25.387 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 791 k 2024-09-16T16:44:25.387 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:44:27.148 INFO:teuthology.orchestra.run.smithi123.stdout:cephadm-19.3.0-4879.g81fedcb3.el9.noarch.rpm 446 kB/s | 785 kB 00:01 2024-09-16T16:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:44:27.149 INFO:teuthology.orchestra.run.smithi123.stdout:Total 445 kB/s | 785 kB 00:01 2024-09-16T16:44:27.150 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:44:27.156 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:44:27.162 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:44:27.163 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:44:27.237 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:44:27.714 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-16T16:44:27.811 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-16T16:44:28.163 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-16T16:44:28.426 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-16T16:44:28.426 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:28.426 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:44:28.426 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:44:28.426 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:28.426 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:44:28.659 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T16:44:29.227 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:03:52 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:44:29.361 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:44:29.361 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:29.361 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-16T16:44:29.361 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:29.361 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:44:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 158 k 2024-09-16T16:44:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:29.362 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:44:29.362 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:29.362 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:44:29.362 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:29.362 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 158 k 2024-09-16T16:44:29.363 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 644 k 2024-09-16T16:44:29.363 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:44:29.937 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-immutable-object-cache-19.3.0-4879.g81fedc 275 kB/s | 158 kB 00:00 2024-09-16T16:44:29.937 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:44:29.937 INFO:teuthology.orchestra.run.smithi123.stdout:Total 275 kB/s | 158 kB 00:00 2024-09-16T16:44:29.938 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:44:29.947 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:44:29.947 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:44:30.004 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:44:30.004 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:44:30.163 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:44:30.262 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-16T16:44:30.288 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-16T16:44:30.288 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:44:30.288 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T16:44:30.288 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:31.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3. 1/1 2024-09-16T16:44:31.208 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:31.208 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:44:31.208 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:44:31.208 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:31.209 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:44:31.464 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr 2024-09-16T16:44:32.045 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:03:55 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:44:32.137 INFO:teuthology.orchestra.run.smithi123.stdout:Package ceph-mgr-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:44:32.181 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:44:32.182 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:44:32.183 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:44:32.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T16:44:32.827 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:03:55 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:44:32.961 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:44:32.962 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:32.962 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:44:32.962 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:32.962 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 14 M 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 34 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 16 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:32.963 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:44:32.964 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:32.964 INFO:teuthology.orchestra.run.smithi123.stdout:Install 11 Packages 2024-09-16T16:44:32.964 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:32.964 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 18 M 2024-09-16T16:44:32.964 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 155 M 2024-09-16T16:44:32.964 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:44:33.473 INFO:teuthology.orchestra.run.smithi123.stdout:(1/11): ceph-grafana-dashboards-19.3.0-4879.g81 86 kB/s | 34 kB 00:00 2024-09-16T16:44:33.498 INFO:teuthology.orchestra.run.smithi123.stdout:(2/11): ceph-prometheus-alerts-19.3.0-4879.g81f 39 kB/s | 16 kB 00:00 2024-09-16T16:44:33.640 INFO:teuthology.orchestra.run.smithi123.stdout:(3/11): python3-protobuf-3.14.0-13.el9.noarch.r 1.6 MB/s | 269 kB 00:00 2024-09-16T16:44:33.690 INFO:teuthology.orchestra.run.smithi123.stdout:(4/11): protobuf-compiler-3.14.0-13.el9.x86_64. 4.4 MB/s | 863 kB 00:00 2024-09-16T16:44:33.741 INFO:teuthology.orchestra.run.smithi123.stdout:(5/11): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.3 MB/s | 548 kB 00:00 2024-09-16T16:44:33.766 INFO:teuthology.orchestra.run.smithi123.stdout:(6/11): grpc-data-1.46.7-10.el9.noarch.rpm 265 kB/s | 19 kB 00:00 2024-09-16T16:44:33.808 INFO:teuthology.orchestra.run.smithi123.stdout:(7/11): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2024-09-16T16:44:33.833 INFO:teuthology.orchestra.run.smithi123.stdout:(8/11): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-16T16:44:33.884 INFO:teuthology.orchestra.run.smithi123.stdout:(9/11): python3-grpcio-1.46.7-10.el9.x86_64.rpm 14 MB/s | 2.0 MB 00:00 2024-09-16T16:44:33.909 INFO:teuthology.orchestra.run.smithi123.stdout:(10/11): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2024-09-16T16:44:36.142 INFO:teuthology.orchestra.run.smithi123.stdout:(11/11): ceph-mgr-dashboard-19.3.0-4879.g81fedc 4.6 MB/s | 14 MB 00:03 2024-09-16T16:44:36.147 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:44:36.147 INFO:teuthology.orchestra.run.smithi123.stdout:Total 5.7 MB/s | 18 MB 00:03 2024-09-16T16:44:36.181 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:44:36.203 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:44:36.203 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:44:36.462 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:44:36.462 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:44:37.023 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:44:37.115 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/11 2024-09-16T16:44:37.146 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/11 2024-09-16T16:44:37.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 3/11 2024-09-16T16:44:37.364 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 4/11 2024-09-16T16:44:37.414 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 5/11 2024-09-16T16:44:37.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 6/11 2024-09-16T16:44:37.550 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 7/11 2024-09-16T16:44:37.579 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 8/11 2024-09-16T16:44:37.611 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9 9/11 2024-09-16T16:44:40.781 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el 10/11 2024-09-16T16:44:40.907 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 11/11 2024-09-16T16:44:42.873 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 11/11 2024-09-16T16:44:42.873 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el 1/11 2024-09-16T16:44:42.873 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noa 2/11 2024-09-16T16:44:42.873 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9 3/11 2024-09-16T16:44:42.873 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 4/11 2024-09-16T16:44:42.873 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 5/11 2024-09-16T16:44:42.874 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 6/11 2024-09-16T16:44:42.874 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 7/11 2024-09-16T16:44:42.874 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 8/11 2024-09-16T16:44:42.874 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/11 2024-09-16T16:44:42.874 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 10/11 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 11/11 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:44:43.183 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:43.184 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:44:43.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T16:44:44.037 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:07 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:44:44.169 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 7.4 M 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T16:44:44.170 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout:Install 12 Packages 2024-09-16T16:44:44.171 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:44.172 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 38 M 2024-09-16T16:44:44.172 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 212 M 2024-09-16T16:44:44.172 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:44:44.439 INFO:teuthology.orchestra.run.smithi123.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-16T16:44:44.490 INFO:teuthology.orchestra.run.smithi123.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.0 MB/s | 801 kB 00:00 2024-09-16T16:44:44.615 INFO:teuthology.orchestra.run.smithi123.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 186 kB/s | 33 kB 00:00 2024-09-16T16:44:44.665 INFO:teuthology.orchestra.run.smithi123.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-09-16T16:44:44.765 INFO:teuthology.orchestra.run.smithi123.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 385 kB/s | 38 kB 00:00 2024-09-16T16:44:45.582 INFO:teuthology.orchestra.run.smithi123.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-09-16T16:44:45.724 INFO:teuthology.orchestra.run.smithi123.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.7 MB/s | 245 kB 00:00 2024-09-16T16:44:46.233 INFO:teuthology.orchestra.run.smithi123.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 3.4 MB/s | 4.9 MB 00:01 2024-09-16T16:44:46.316 INFO:teuthology.orchestra.run.smithi123.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-16T16:44:46.675 INFO:teuthology.orchestra.run.smithi123.stdout:(10/12): ceph-mgr-diskprediction-local-19.3.0-4 3.1 MB/s | 7.4 MB 00:02 2024-09-16T16:44:46.934 INFO:teuthology.orchestra.run.smithi123.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-16T16:44:48.292 INFO:teuthology.orchestra.run.smithi123.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.2 MB/s | 16 MB 00:01 2024-09-16T16:44:48.293 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:44:48.293 INFO:teuthology.orchestra.run.smithi123.stdout:Total 9.3 MB/s | 38 MB 00:04 2024-09-16T16:44:48.649 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:44:48.674 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:44:49.125 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:44:49.126 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:44:49.900 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:44:49.974 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T16:44:50.023 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T16:44:50.132 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T16:44:50.175 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T16:44:50.359 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T16:44:50.390 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T16:44:50.476 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T16:44:51.125 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T16:44:51.245 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T16:44:52.480 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T16:44:53.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T16:44:53.581 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 12/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 12/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fed 1/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T16:44:56.313 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T16:44:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T16:44:56.740 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T16:44:56.741 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T16:44:56.742 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:56.742 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:44:57.160 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-rook 2024-09-16T16:44:57.736 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:20 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:44:57.870 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:44:57.871 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:57.871 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:44:57.871 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 49 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T16:44:57.872 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout:Install 13 Packages 2024-09-16T16:44:57.873 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:44:57.874 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 2.2 M 2024-09-16T16:44:57.874 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 26 M 2024-09-16T16:44:57.874 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:44:58.064 INFO:teuthology.orchestra.run.smithi123.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 226 kB/s | 26 kB 00:00 2024-09-16T16:44:58.089 INFO:teuthology.orchestra.run.smithi123.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 138 kB/s | 19 kB 00:00 2024-09-16T16:44:58.156 INFO:teuthology.orchestra.run.smithi123.stdout:(3/13): ceph-mgr-rook-19.3.0-4879.g81fedcb3.el9 239 kB/s | 49 kB 00:00 2024-09-16T16:44:58.206 INFO:teuthology.orchestra.run.smithi123.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.9 MB/s | 222 kB 00:00 2024-09-16T16:44:58.231 INFO:teuthology.orchestra.run.smithi123.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-09-16T16:44:58.282 INFO:teuthology.orchestra.run.smithi123.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 644 kB/s | 32 kB 00:00 2024-09-16T16:44:58.315 INFO:teuthology.orchestra.run.smithi123.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 639 kB/s | 54 kB 00:00 2024-09-16T16:44:58.332 INFO:teuthology.orchestra.run.smithi123.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-16T16:44:58.366 INFO:teuthology.orchestra.run.smithi123.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.3 MB/s | 279 kB 00:00 2024-09-16T16:44:58.399 INFO:teuthology.orchestra.run.smithi123.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.6 MB/s | 220 kB 00:00 2024-09-16T16:44:58.425 INFO:teuthology.orchestra.run.smithi123.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.7 MB/s | 90 kB 00:00 2024-09-16T16:44:58.458 INFO:teuthology.orchestra.run.smithi123.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.6 MB/s | 1.0 MB 00:00 2024-09-16T16:44:58.475 INFO:teuthology.orchestra.run.smithi123.stdout:(13/13): python3-rsa-4.9-2.el9.noarch.rpm 550 kB/s | 59 kB 00:00 2024-09-16T16:44:58.478 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:44:58.479 INFO:teuthology.orchestra.run.smithi123.stdout:Total 3.7 MB/s | 2.2 MB 00:00 2024-09-16T16:44:58.542 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:44:58.558 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:44:58.558 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:44:58.646 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:44:58.648 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:44:59.160 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:44:59.336 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T16:44:59.393 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T16:44:59.456 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T16:44:59.485 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T16:44:59.527 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T16:44:59.627 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T16:44:59.729 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T16:44:59.778 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T16:45:00.489 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T16:45:00.534 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T16:45:00.567 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T16:45:00.615 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T16:45:00.637 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 13/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 13/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 1/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T16:45:01.398 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T16:45:01.399 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T16:45:01.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:01.763 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:02.034 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T16:45:02.601 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:25 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:02.733 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:02.733 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:02.733 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 154 k 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:02.734 INFO:teuthology.orchestra.run.smithi123.stdout:Install 6 Packages 2024-09-16T16:45:02.735 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:02.735 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 7.0 M 2024-09-16T16:45:02.735 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 33 M 2024-09-16T16:45:02.735 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:03.110 INFO:teuthology.orchestra.run.smithi123.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-16T16:45:03.135 INFO:teuthology.orchestra.run.smithi123.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4879.g81fedcb3.e 661 kB/s | 154 kB 00:00 2024-09-16T16:45:03.193 INFO:teuthology.orchestra.run.smithi123.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 992 kB/s | 58 kB 00:00 2024-09-16T16:45:03.227 INFO:teuthology.orchestra.run.smithi123.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-16T16:45:03.252 INFO:teuthology.orchestra.run.smithi123.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.8 MB/s | 548 kB 00:00 2024-09-16T16:45:04.469 INFO:teuthology.orchestra.run.smithi123.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.8 MB/s | 6.0 MB 00:01 2024-09-16T16:45:04.471 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:04.472 INFO:teuthology.orchestra.run.smithi123.stdout:Total 4.0 MB/s | 7.0 MB 00:01 2024-09-16T16:45:04.543 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:04.554 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:04.554 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:04.655 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:04.655 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:04.972 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:05.133 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T16:45:05.212 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T16:45:05.730 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T16:45:05.815 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T16:45:05.883 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T16:45:05.908 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 6/6 2024-09-16T16:45:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 6/6 2024-09-16T16:45:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 1/6 2024-09-16T16:45:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T16:45:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T16:45:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T16:45:06.734 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T16:45:07.011 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T16:45:07.012 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T16:45:07.012 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:07.012 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:07.323 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-fuse 2024-09-16T16:45:07.890 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:30 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:08.022 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 884 k 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:08.023 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:08.024 INFO:teuthology.orchestra.run.smithi123.stdout:Install 2 Packages 2024-09-16T16:45:08.024 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:08.024 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 964 k 2024-09-16T16:45:08.024 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 4.1 M 2024-09-16T16:45:08.024 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:08.229 INFO:teuthology.orchestra.run.smithi123.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 533 kB/s | 80 kB 00:00 2024-09-16T16:45:08.554 INFO:teuthology.orchestra.run.smithi123.stdout:(2/2): ceph-fuse-19.3.0-4879.g81fedcb3.el9.x86_ 1.8 MB/s | 884 kB 00:00 2024-09-16T16:45:08.555 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:08.555 INFO:teuthology.orchestra.run.smithi123.stdout:Total 1.8 MB/s | 964 kB 00:00 2024-09-16T16:45:08.567 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:08.591 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:08.591 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:08.671 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:08.672 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:08.925 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:09.081 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T16:45:09.153 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/2 2024-09-16T16:45:09.866 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2/2 2024-09-16T16:45:09.867 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/2 2024-09-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:10.139 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:10.140 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:10.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-volume 2024-09-16T16:45:11.028 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:34 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:11.164 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.3.0-4879.g81fedcb3.el9 ceph-noarch 285 k 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 285 k 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 1.3 M 2024-09-16T16:45:11.165 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:11.617 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-volume-19.3.0-4879.g81fedcb3.el9.noarch.rp 632 kB/s | 285 kB 00:00 2024-09-16T16:45:11.617 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:11.618 INFO:teuthology.orchestra.run.smithi123.stdout:Total 629 kB/s | 285 kB 00:00 2024-09-16T16:45:11.618 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:11.627 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:11.627 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:11.657 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:11.657 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:11.882 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:11.961 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-16T16:45:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-16T16:45:11.987 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:45:11.987 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T16:45:11.987 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:13.043 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 1/1 2024-09-16T16:45:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:13.044 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.3.0-4879.g81fedcb3.el9.noarch 2024-09-16T16:45:13.044 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:13.044 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:13.350 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados-devel 2024-09-16T16:45:13.928 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:36 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:14.062 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 125 k 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 125 k 2024-09-16T16:45:14.063 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 445 k 2024-09-16T16:45:14.064 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:14.472 INFO:teuthology.orchestra.run.smithi123.stdout:librados-devel-19.3.0-4879.g81fedcb3.el9.x86_64 306 kB/s | 125 kB 00:00 2024-09-16T16:45:14.472 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:14.473 INFO:teuthology.orchestra.run.smithi123.stdout:Total 304 kB/s | 125 kB 00:00 2024-09-16T16:45:14.473 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:14.478 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:14.479 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:14.543 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:14.543 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:14.716 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:14.811 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:15.255 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:15.536 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:45:15.536 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:15.536 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:15.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs2 2024-09-16T16:45:16.401 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:39 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:16.492 INFO:teuthology.orchestra.run.smithi123.stdout:Package libcephfs2-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:45:16.535 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:16.537 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:45:16.537 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:16.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install libcephfs-devel 2024-09-16T16:45:17.180 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:40 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:17.313 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:17.313 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 32 k 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:45:17.314 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:17.315 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 32 k 2024-09-16T16:45:17.315 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 142 k 2024-09-16T16:45:17.315 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:17.592 INFO:teuthology.orchestra.run.smithi123.stdout:libcephfs-devel-19.3.0-4879.g81fedcb3.el9.x86_6 114 kB/s | 32 kB 00:00 2024-09-16T16:45:17.593 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:17.593 INFO:teuthology.orchestra.run.smithi123.stdout:Total 114 kB/s | 32 kB 00:00 2024-09-16T16:45:17.593 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:17.596 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:17.596 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:17.616 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:17.616 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:17.697 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:17.806 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:18.195 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:18.441 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:18.441 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:18.441 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:18.442 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:45:18.442 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:18.442 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:18.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librados2 2024-09-16T16:45:19.237 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:42 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:19.327 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:45:19.370 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:19.371 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:45:19.371 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:19.447 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install librbd1 2024-09-16T16:45:20.011 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:43 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:20.101 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:45:20.145 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:20.146 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:45:20.146 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:20.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rados 2024-09-16T16:45:20.783 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:43 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:20.873 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rados-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:45:20.916 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:20.917 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:45:20.917 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:20.989 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rgw 2024-09-16T16:45:21.546 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:44 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:21.635 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rgw-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:45:21.678 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:21.679 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:45:21.679 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:21.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-cephfs 2024-09-16T16:45:22.349 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:45 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:22.439 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-cephfs-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:45:22.483 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:22.484 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:45:22.484 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:22.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install python3-rbd 2024-09-16T16:45:23.127 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:46 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:23.219 INFO:teuthology.orchestra.run.smithi123.stdout:Package python3-rbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 is already installed. 2024-09-16T16:45:23.263 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:23.264 INFO:teuthology.orchestra.run.smithi123.stdout:Nothing to do. 2024-09-16T16:45:23.264 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:23.334 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-fuse 2024-09-16T16:45:23.896 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:46 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:24.029 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:24.029 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 91 k 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 91 k 2024-09-16T16:45:24.030 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 321 k 2024-09-16T16:45:24.031 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:24.436 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-fuse-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 224 kB/s | 91 kB 00:00 2024-09-16T16:45:24.437 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:24.437 INFO:teuthology.orchestra.run.smithi123.stdout:Total 223 kB/s | 91 kB 00:00 2024-09-16T16:45:24.437 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:24.447 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:24.447 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:24.511 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:24.511 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:24.658 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:24.759 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:25.155 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:25.424 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:25.424 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:25.424 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:25.424 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:45:25.425 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:25.425 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:25.723 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-mirror 2024-09-16T16:45:26.295 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:49 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:26.430 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:26.431 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:26.431 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repo Size 2024-09-16T16:45:26.431 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:26.432 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:26.432 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 3.5 M 2024-09-16T16:45:26.432 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:26.432 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:26.432 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:26.432 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:45:26.433 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:26.433 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 3.5 M 2024-09-16T16:45:26.433 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 30 M 2024-09-16T16:45:26.433 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:27.439 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-mirror-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 3.5 MB/s | 3.5 MB 00:01 2024-09-16T16:45:27.440 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:27.440 INFO:teuthology.orchestra.run.smithi123.stdout:Total 3.5 MB/s | 3.5 MB 00:01 2024-09-16T16:45:27.440 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:27.452 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:27.452 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:27.538 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:27.538 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:28.130 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:28.198 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:28.223 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:28.223 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T16:45:28.224 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T16:45:28.224 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T16:45:28.224 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T16:45:28.224 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:29.467 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:29.467 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:29.467 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:29.467 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:45:29.468 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:29.468 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:29.746 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install rbd-nbd 2024-09-16T16:45:30.331 INFO:teuthology.orchestra.run.smithi123.stdout:Last metadata expiration check: 0:04:53 ago on Mon 16 Sep 2024 04:40:37 PM UTC. 2024-09-16T16:45:30.464 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-16T16:45:30.464 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:30.464 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-09-16T16:45:30.464 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:30.465 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-16T16:45:30.465 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.3.0-4879.g81fedcb3.el9 ceph 177 k 2024-09-16T16:45:30.465 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:30.465 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-16T16:45:30.465 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-09-16T16:45:30.465 INFO:teuthology.orchestra.run.smithi123.stdout:Install 1 Package 2024-09-16T16:45:30.465 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:30.466 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 177 k 2024-09-16T16:45:30.466 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 685 k 2024-09-16T16:45:30.466 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-16T16:45:30.675 INFO:teuthology.orchestra.run.smithi123.stdout:rbd-nbd-19.3.0-4879.g81fedcb3.el9.x86_64.rpm 843 kB/s | 177 kB 00:00 2024-09-16T16:45:30.676 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-16T16:45:30.676 INFO:teuthology.orchestra.run.smithi123.stdout:Total 836 kB/s | 177 kB 00:00 2024-09-16T16:45:30.677 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-16T16:45:30.686 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-16T16:45:30.686 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-16T16:45:30.757 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-16T16:45:30.757 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-16T16:45:30.933 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-16T16:45:31.031 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:31.467 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 1/1 2024-09-16T16:45:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:31.725 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-16T16:45:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.3.0-4879.g81fedcb3.el9.x86_64 2024-09-16T16:45:31.725 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:31.725 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-16T16:45:31.983 DEBUG:teuthology.parallel:result is None 2024-09-16T16:45:31.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F9%2Fx86_64&sha1=81fedcb3b484c843d22ffbc8d179cb567b7083b5 2024-09-16T16:45:32.118 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T16:45:32.140 INFO:teuthology.orchestra.run.smithi123.stdout:19.3.0-4879.g81fedcb3.el9 2024-09-16T16:45:32.140 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4879.g81fedcb3.el9 2024-09-16T16:45:32.141 INFO:teuthology.task.install:The correct ceph version 19.3.0-4879.g81fedcb3 is installed. 2024-09-16T16:45:32.142 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T16:45:32.142 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:32.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T16:45:32.206 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T16:45:32.206 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:32.206 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T16:45:32.270 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T16:45:32.334 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T16:45:32.335 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:32.335 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T16:45:32.399 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T16:45:32.463 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T16:45:32.463 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:32.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T16:45:32.527 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T16:45:32.591 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T16:45:32.668 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T16:45:32.669 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2024-09-16T16:45:32.693 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T16:45:32.693 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T16:45:32.795 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T16:45:32.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T16:45:32.826 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T16:45:32.826 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr stats period': 30}, 'osd.0': {'crimson seastar cpu cores': '0-1'}, 'osd.1': {'crimson seastar cpu cores': '2-3'}, 'osd.2': {'crimson seastar cpu cores': '4-5'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(POOL_FULL\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T16:45:32.826 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7907389', 'branch': 'wip-matanb-crimson-testing-59543', 'description': 'crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore tasks/rbd_api_tests}', 'email': None, 'first_in_suite': False, 'job_id': '7907389', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-matanb-crimson-testing-59543'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr stats period': 30}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'osd.0': {'crimson seastar cpu cores': '0-1'}, 'osd.1': {'crimson seastar cpu cores': '2-3'}, 'osd.2': {'crimson seastar cpu cores': '4-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(POOL_FULL\\)'], 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779'}}, 'owner': 'scheduled_matan@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 6678, 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_0325fd3c11a225c0fd9eb38fab88aad1370ad779/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '0325fd3c11a225c0fd9eb38fab88aad1370ad779', 'targets': {'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCoOyiajz1larOvU1hiCsPOS5mx8iADdKwpvxxMg3tEZEscWuBWp5PL+KdhS1za8sPAgi3eHFtdRCfs6HFKWImA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr stats period': 30}, 'osd.0': {'crimson seastar cpu cores': '0-1'}, 'osd.1': {'crimson seastar cpu cores': '2-3'}, 'osd.2': {'crimson seastar cpu cores': '4-5'}}, 'flavor': 'crimson', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(POOL_FULL\\)'], 'sha1': '81fedcb3b484c843d22ffbc8d179cb567b7083b5', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rbd/crimson/test_crimson_librbd.sh']}, 'env': {'RBD_FEATURES': '61'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_16:23:48', 'tube': 'smithi', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T16:45:32.826 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T16:45:32.875 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T16:45:32.940 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:32.940 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T16:45:32.995 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T16:45:32.996 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-16 16:45:28.474616147 +0000 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-16 16:39:45.314549814 +0000 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-16 16:39:45.314549814 +0000 2024-09-16T16:45:33.051 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-16 16:39:45.314549814 +0000 2024-09-16T16:45:33.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T16:45:33.114 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-16T16:45:33.114 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-16T16:45:33.115 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000204687 s, 2.5 MB/s 2024-09-16T16:45:33.116 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T16:45:33.172 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-09-16T16:45:33.227 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T16:45:33.227 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T16:45:33.227 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-16T16:45:33.228 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T16:45:33.228 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T16:45:33.228 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-16 16:45:28.475616123 +0000 2024-09-16T16:45:33.228 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-16 16:39:45.646541852 +0000 2024-09-16T16:45:33.228 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-16 16:39:45.646541852 +0000 2024-09-16T16:45:33.228 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-16 16:39:45.646541852 +0000 2024-09-16T16:45:33.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T16:45:33.290 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-16T16:45:33.290 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-16T16:45:33.290 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000170155 s, 3.0 MB/s 2024-09-16T16:45:33.291 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T16:45:33.348 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-09-16T16:45:33.403 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T16:45:33.403 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T16:45:33.403 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T16:45:33.403 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T16:45:33.404 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T16:45:33.404 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-16 16:45:28.475616123 +0000 2024-09-16T16:45:33.404 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-16 16:39:45.966534178 +0000 2024-09-16T16:45:33.404 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-16 16:39:45.966534178 +0000 2024-09-16T16:45:33.404 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-16 16:39:45.966534178 +0000 2024-09-16T16:45:33.404 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T16:45:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-16T16:45:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-16T16:45:33.466 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00015512 s, 3.3 MB/s 2024-09-16T16:45:33.467 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T16:45:33.523 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-09-16T16:45:33.578 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T16:45:33.578 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T16:45:33.578 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T16:45:33.578 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T16:45:33.578 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T16:45:33.578 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-16 16:45:28.475616123 +0000 2024-09-16T16:45:33.578 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-16 16:39:46.302526121 +0000 2024-09-16T16:45:33.579 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-16 16:39:46.302526121 +0000 2024-09-16T16:45:33.579 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-16 16:39:46.302526121 +0000 2024-09-16T16:45:33.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T16:45:33.641 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-16T16:45:33.641 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-16T16:45:33.641 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000241258 s, 2.1 MB/s 2024-09-16T16:45:33.643 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T16:45:33.699 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T16:45:33.699 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-16T16:45:33.699 INFO:tasks.ceph:Generating config... 2024-09-16T16:45:33.700 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-16T16:45:33.700 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-16T16:45:33.700 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-16T16:45:33.700 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-16T16:45:33.701 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-16T16:45:33.701 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T16:45:33.701 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T16:45:33.701 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T16:45:33.701 INFO:tasks.ceph:[mon] osd pool default crimson = True 2024-09-16T16:45:33.701 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-16T16:45:33.701 INFO:tasks.ceph:[osd] crimson alien thread cpu cores = 6-7 2024-09-16T16:45:33.701 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2024-09-16T16:45:33.701 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T16:45:33.701 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T16:45:33.701 INFO:tasks.ceph:[osd] osd objectstore = seastore 2024-09-16T16:45:33.702 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T16:45:33.702 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2024-09-16T16:45:33.702 INFO:tasks.ceph:[global] ms client mode = crc 2024-09-16T16:45:33.702 INFO:tasks.ceph:[global] ms cluster mode = crc 2024-09-16T16:45:33.702 INFO:tasks.ceph:[global] ms mon client mode = crc 2024-09-16T16:45:33.702 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2024-09-16T16:45:33.702 INFO:tasks.ceph:[global] ms mon service mode = crc 2024-09-16T16:45:33.702 INFO:tasks.ceph:[global] ms service mode = crc 2024-09-16T16:45:33.702 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T16:45:33.702 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T16:45:33.702 INFO:tasks.ceph:[mgr] mgr stats period = 30 2024-09-16T16:45:33.702 INFO:tasks.ceph:[osd.0] crimson seastar cpu cores = 0-1 2024-09-16T16:45:33.702 INFO:tasks.ceph:[osd.1] crimson seastar cpu cores = 2-3 2024-09-16T16:45:33.703 INFO:tasks.ceph:[osd.2] crimson seastar cpu cores = 4-5 2024-09-16T16:45:33.703 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T16:45:33.703 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T16:45:33.805 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T16:45:33.808 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T16:45:33.859 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T16:45:33.883 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.123')] 2024-09-16T16:45:33.883 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.123', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson alien thread cpu cores': '6-7', 'crimson osd obc lru size': 10, 'debug ms': 1, 'debug osd': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'mgr stats period': 30}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'osd.0': {'crimson seastar cpu cores': '0-1'}, 'osd.1': {'crimson seastar cpu cores': '2-3'}, 'osd.2': {'crimson seastar cpu cores': '4-5'}} 2024-09-16T16:45:33.884 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:33.884 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T16:45:33.938 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.123 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T16:45:34.029 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:34.029+0000 7fa0b2257f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:34.029+0000 7fa0b2257f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:34.029+0000 7fa0b2257f00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: generated fsid 0d2cd943-ff49-4927-aa46-a373e0b8c2c2 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = quincy 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 0 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stdout:fsid 0d2cd943-ff49-4927-aa46-a373e0b8c2c2 2024-09-16T16:45:34.030 INFO:teuthology.orchestra.run.smithi123.stdout:last_changed 2024-09-16T16:45:34.030807+0000 2024-09-16T16:45:34.031 INFO:teuthology.orchestra.run.smithi123.stdout:created 2024-09-16T16:45:34.030807+0000 2024-09-16T16:45:34.031 INFO:teuthology.orchestra.run.smithi123.stdout:min_mon_release 17 (quincy) 2024-09-16T16:45:34.031 INFO:teuthology.orchestra.run.smithi123.stdout:election_strategy: 1 2024-09-16T16:45:34.031 INFO:teuthology.orchestra.run.smithi123.stdout:0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2024-09-16T16:45:34.031 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-16T16:45:34.032 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T16:45:34.047 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0d2cd943-ff49-4927-aa46-a373e0b8c2c2... 2024-09-16T16:45:34.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T16:45:34.128 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2024-09-16T16:45:34.129 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2024-09-16T16:45:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2024-09-16T16:45:34.131 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2024-09-16T16:45:34.131 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2024-09-16T16:45:34.131 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2024-09-16T16:45:34.131 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2024-09-16T16:45:34.131 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2024-09-16T16:45:34.131 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2024-09-16T16:45:34.132 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = 172.21.15.123 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: enable experimental unrecoverable data corrupting features = crimson 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: ms client mode = crc 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: ms cluster mode = crc 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: ms mon client mode = crc 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: ms mon cluster mode = crc 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: ms mon service mode = crc 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: ms service mode = crc 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 0d2cd943-ff49-4927-aa46-a373e0b8c2c2 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2024-09-16T16:45:34.133 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug trim objects = true 2024-09-16T16:45:34.134 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2024-09-16T16:45:34.135 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: debug monc = 20 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: crimson alien thread cpu cores = 6-7 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: crimson osd obc lru size = 10 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = seastore 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: osd shutdown pgref assert = True 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-09-16T16:45:34.136 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mgr stats period = 30 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2024-09-16T16:45:34.137 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2024-09-16T16:45:34.138 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: mon min osdmap epochs = 50 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: mon osdmap full prune interval = 2 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: mon osdmap full prune min = 15 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: mon osdmap full prune txsize = 2 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: paxos service trim min = 10 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default crimson = True 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2024-09-16T16:45:34.139 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout: crimson seastar cpu cores = 0-1 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout: crimson seastar cpu cores = 2-3 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-09-16T16:45:34.140 INFO:teuthology.orchestra.run.smithi123.stdout: crimson seastar cpu cores = 4-5 2024-09-16T16:45:34.141 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T16:45:34.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T16:45:34.246 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T16:45:34.247 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:34.247 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T16:45:34.262 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:34.262 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T16:45:34.317 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T16:45:34.317 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:34.317 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T16:45:34.317 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T16:45:34.391 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:34.391 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T16:45:34.446 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T16:45:34.446 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T16:45:34.446 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T16:45:34.545 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T16:45:34.548 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T16:45:34.548 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T16:45:34.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T16:45:34.597 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T16:45:34.610 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T16:45:34.610 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-16T16:45:34.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T16:45:34.634 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T16:45:34.635 INFO:tasks.ceph:role: osd.0 2024-09-16T16:45:34.635 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T16:45:34.635 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T16:45:34.706 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T16:45:34.706 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T16:45:34.707 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T16:45:35.222 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-16T16:45:35.224 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-16T16:45:35.224 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T16:45:35.373 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T16:45:35.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T16:45:35.470 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T16:45:35.470 INFO:tasks.ceph:role: osd.1 2024-09-16T16:45:35.470 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T16:45:35.470 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T16:45:35.542 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T16:45:35.542 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T16:45:35.542 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T16:45:35.542 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T16:45:35.543 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T16:45:35.543 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-16T16:45:35.543 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T16:45:35.543 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T16:45:35.543 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T16:45:35.543 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T16:45:36.028 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-16T16:45:36.031 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-16T16:45:36.031 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T16:45:36.120 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T16:45:36.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T16:45:36.213 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T16:45:36.213 INFO:tasks.ceph:role: osd.2 2024-09-16T16:45:36.213 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T16:45:36.214 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T16:45:36.285 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T16:45:36.286 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T16:45:36.810 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2024-09-16T16:45:36.812 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2024-09-16T16:45:36.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T16:45:36.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T16:45:36.982 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T16:45:37.284 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-16T16:45:37.285 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:37,286 seastar - Reactor backend: linux-aio 2024-09-16T16:45:37.300 INFO:teuthology.orchestra.run.smithi123.stderr:==27656==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-16T16:45:37.406 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0xc92c1 0xc9e39 0x36084f8 0x396d169 0x45855a2 0x4621f13 0x5fdbff1 0x5fdc222 0x5fb92bb 0x5e00fef 0x5e19231 0x71579d2 0x71691a5 0x716a642 0x716a6c7 0x716a7c9 0x716a8c1 0x716aace 0x716acb4 0x716b003 0x716b3a7 0x716b5d7 0x716b841 0x716bb0e 0x716bd2c 0x716bde4 0x716beaa 0x716bf8b 0x5c63e30 0x5c63f3a 0x5c63ff6 0x5c640b2 0x5c6416e 0x5c642bb 0x5c64446 0x5c64807 0x5c7e60a 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3962bc0 0x396439a 0x3804c28 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:37.406 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:37.416 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:37,416 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-16T16:45:37.416 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:37,416 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 0-1 2024-09-16T16:45:37.416 INFO:teuthology.orchestra.run.smithi123.stderr:DEBUG 2024-09-16 16:45:37,417 [shard 0:main] seastar - reactor::drain 2024-09-16T16:45:37.930 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:37.930 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-16T16:45:37.930 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-16T16:45:37.930 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-16T16:45:37.931 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:37.931 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T16:45:37.987 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-16T16:45:37.990 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:37,991 seastar - Reactor backend: linux-aio 2024-09-16T16:45:37.998 INFO:teuthology.orchestra.run.smithi123.stderr:==27654==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-16T16:45:38.279 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x162b9f 0x422c4 0x3ef7b 0xfd531 0xb85348d 0xba3392c 0xba33a5e 0xba33bda 0xba34389 0xb9ceb14 0x9843791 0x9af9974 0x9b11b72 0x99a2168 0x9942951 0x927469d 0x932dfed 0x932ef8c 0x932f00f 0x932f094 0x932f196 0x932f28e 0x932f314 0x932f3cd 0x932f583 0x932f6bb 0x932f736 0x932f7bb 0x932f916 0x932fa30 0x932fab5 0x932fb93 0x9552886 0x961b5f3 0x961b770 0x96dc4d1 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:38.279 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff9da8c7a4 0xffffffff9e6d1d62 0xffffffff9e800bb2 2024-09-16T16:45:38.427 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x4ab09 0xb9e9103 0xb9ea53a 0xb9ea7b4 0xb9ea95c 0xba022df 0xba313d1 0xba31a18 0xb9d8ccf 0xb9dc6f8 0xb9c7696 0xb9c7a7a 0x9afae24 0x9b11b72 0x99a2168 0x9942951 0x927469d 0x9274db2 0x92752ce 0x9275457 0x927569e 0x9275976 0x9275a2b 0x9592ce8 0x9592f4d 0x95caf33 0x96cf51e 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:38.427 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:38.500 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542d8a 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:38.501 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:38.558 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x4ab7c 0x5542dfb 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:38.558 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:38.570 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x55427a2 0x9e3397b 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:38.570 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0xdd2e1 0x4ab1c 0x5542dfb 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:38.666 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x4ab79 0x5542dfb 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:38.666 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:38.735 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542c26 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:38.736 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:39.118 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x357849f 0x5542dfb 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:39.118 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542726 0x9e3397b 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:39.176 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:40.161 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:40.161 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-16T16:45:40.161 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-16T16:45:40.161 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-16T16:45:40.161 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:40.161 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T16:45:40.185 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T16:45:40.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T16:45:40.487 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-16T16:45:40.488 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:40,488 seastar - Reactor backend: linux-aio 2024-09-16T16:45:40.493 INFO:teuthology.orchestra.run.smithi123.stderr:==27701==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-16T16:45:40.600 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5fb8dfc 0x5fb923a 0x5e00fef 0x5e19231 0x71579d2 0x71691a5 0x716a642 0x716a6c7 0x716a7c9 0x716a8c1 0x716aace 0x716acb4 0x716b003 0x716b3a7 0x716b5d7 0x716b841 0x716bb0e 0x716bd2c 0x716bde4 0x716beaa 0x716bf8b 0x5c63e30 0x5c63f3a 0x5c63ff6 0x5c640b2 0x5c6416e 0x5c642bb 0x5c64446 0x5c64807 0x5c7e60a 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3962bc0 0x396439a 0x3804c28 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:40.600 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:40.609 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:40,610 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-16T16:45:40.609 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:40,610 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 2-3 2024-09-16T16:45:40.610 INFO:teuthology.orchestra.run.smithi123.stderr:DEBUG 2024-09-16 16:45:40,611 [shard 0:main] seastar - reactor::drain 2024-09-16T16:45:41.114 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:41.114 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-16T16:45:41.115 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-16T16:45:41.115 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-16T16:45:41.115 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:41.115 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T16:45:41.175 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-16T16:45:41.179 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:41,179 seastar - Reactor backend: linux-aio 2024-09-16T16:45:41.197 INFO:teuthology.orchestra.run.smithi123.stderr:==27699==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-16T16:45:41.473 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x4abd0 0x61146f4 0xba3480f 0xba34940 0xba34b2c 0xba34cc8 0xba3505e 0xba35270 0xb9d20ef 0xb9d2514 0xb9d84d7 0xb9dc6f8 0xb9c7696 0xb9c7a7a 0x9afa023 0x9b11b72 0x99a2168 0x9942951 0x927469d 0x932dfed 0x932ef8c 0x932f00f 0x932f094 0x932f196 0x932f28e 0x932f314 0x932f3cd 0x932f583 0x932f6bb 0x932f736 0x932f7bb 0x932f916 0x932fa30 0x932fab5 0x932fb93 0x9552886 0x961b5f3 0x961b770 0x96dc4d1 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:41.473 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x954d9bb 0x95e38c3 0x95e3dd5 0x9afac9f 0x9b11b72 0x99a2168 0x9942951 0x927469d 0x9274db2 0x92752ce 0x9275457 0x927569e 0x9275976 0x9275a2b 0x9592ce8 0x9592f4d 0x95caf33 0x96cf51e 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:41.531 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:41.676 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5541dd2 0x5542d9c 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:41.676 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:41.733 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x133c64 0xdd332 0x4ab1c 0x5542dfb 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:41.733 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x133cd1 0xdd332 0x4ab1c 0x5542dfb 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:41.763 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:41.820 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542db9 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:41.820 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:41.841 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x9e3397b 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:41.841 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:41.928 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x9e33934 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:41.928 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x156cf 0xdd312 0x4ab1c 0x5c90278 0xb9ea7a2 0xb9ea95c 0xb9ec717 0xb9ecb62 0xb9d8a8c 0xb9dc6f8 0xb9c7696 0xb9c7a7a 0x9afa023 0x9b11b72 0x99a2168 0x9942951 0x927469d 0x9311f16 0x93129ba 0x9312a3d 0x9312ac2 0x9312bc4 0x9312cbc 0x9312d42 0x9312dfb 0x9312fb1 0x93130e9 0x9313164 0x93131e9 0x9313344 0x931345e 0x93134e3 0x93135c1 0x9552886 0x961b5f3 0x961b770 0x96dc4d1 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:42.241 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:42.348 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542300 0x9e3396e 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:42.348 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:43.341 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:43.341 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-16T16:45:43.341 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-16T16:45:43.341 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-16T16:45:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:43.342 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T16:45:43.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T16:45:43.390 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T16:45:43.677 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-16T16:45:43.678 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:43,678 seastar - Reactor backend: linux-aio 2024-09-16T16:45:43.683 INFO:teuthology.orchestra.run.smithi123.stderr:==27746==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-16T16:45:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x3817d23 0x4621ee0 0x5fdbff1 0x5fdc222 0x5fb92bb 0x5e00fef 0x5e19231 0x71579d2 0x71691a5 0x716a642 0x716a6c7 0x716a7c9 0x716a8c1 0x716aace 0x716acb4 0x716b003 0x716b3a7 0x716b5d7 0x716b841 0x716bb0e 0x716bd2c 0x716bde4 0x716beaa 0x716bf8b 0x5c63e30 0x5c63f3a 0x5c63ff6 0x5c640b2 0x5c6416e 0x5c642bb 0x5c64446 0x5c64807 0x5c7e60a 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3962bc0 0x396439a 0x3804c28 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:43.791 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:43.802 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:43,803 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2024-09-16T16:45:43.802 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:43,803 [shard 0:main] osd - get_early_config: set --thread-affinity 1 --cpuset 4-5 2024-09-16T16:45:43.803 INFO:teuthology.orchestra.run.smithi123.stderr:DEBUG 2024-09-16 16:45:43,804 [shard 0:main] seastar - reactor::drain 2024-09-16T16:45:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-16T16:45:44.306 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-16T16:45:44.306 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-16T16:45:44.306 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:44.306 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T16:45:44.363 INFO:teuthology.orchestra.run.smithi123.stderr:WARNING: debug mode. Not for benchmarking or production 2024-09-16T16:45:44.367 INFO:teuthology.orchestra.run.smithi123.stderr:INFO 2024-09-16 16:45:44,367 seastar - Reactor backend: linux-aio 2024-09-16T16:45:44.383 INFO:teuthology.orchestra.run.smithi123.stderr:==27744==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2024-09-16T16:45:44.448 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0xdd323 0x4b14c 0x381b234 0x5fdd423 0x5fdd8fa 0x6038b24 0x5fb8d06 0x5fb8ee0 0x5fb946c 0x5e00fef 0x5e19231 0x71579d2 0x71691a5 0x716a642 0x716a6c7 0x716a7c9 0x716a8c1 0x716aace 0x716acb4 0x716b003 0x716b3a7 0x716b5d7 0x716b841 0x716bb0e 0x716bd2c 0x716bde4 0x716beaa 0x716bf8b 0x5c63e30 0x5c63f3a 0x5c63ff6 0x5c640b2 0x5c6416e 0x5c642bb 0x5c64446 0x5c64807 0x5c7e60a 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:44.449 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:44.658 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x4218a 0x3ef7b 0xfd531 0xb85348d 0xba3392c 0xba33a5e 0xba33bda 0xba34389 0xb9ceb14 0x9843791 0x9af9f5a 0x9b11b72 0x99a2168 0x9942951 0x927469d 0x932dfed 0x932ef8c 0x932f00f 0x932f094 0x932f196 0x932f28e 0x932f314 0x932f3cd 0x932f583 0x932f6bb 0x932f736 0x932f7bb 0x932f916 0x932fa30 0x932fab5 0x932fb93 0x9552886 0x961b5f3 0x961b770 0x96dc4d1 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:44.658 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:44.858 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542001 0x9e33959 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:44.858 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:44.916 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5541dd2 0x5542d9c 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:44.939 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:44.962 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x3ad3f 0x10d76c 0xdd31a 0x4ab1c 0x5542dfb 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:44.962 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 122 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542d84 0x9e33940 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:45.019 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:45.023 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x554277b 0x9e3397b 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:45.023 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:45.127 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 230 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x5542113 0x9e33959 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:45.127 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:45.423 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0xba1bb81 0xba1c1ac 0xba35301 0xb9d20ef 0xb9d2514 0xb9d84d7 0xb9dc6f8 0xb9c7696 0xb9c7a7a 0x9afa023 0x9b11b72 0x99a2168 0x9942951 0x927469d 0x9311f16 0x93129ba 0x9312a3d 0x9312ac2 0x9312bc4 0x9312cbc 0x9312d42 0x9312dfb 0x9312fb1 0x93130e9 0x9313164 0x93131e9 0x9313344 0x931345e 0x93134e3 0x93135c1 0x9552886 0x961b5f3 0x961b770 0x96dc4d1 0xbb95287 0xbbad105 0xbc5a9af 0xbc6cfab 0xbc6d62d 0x7b30c45 0xc602d44 0x5e669 0x89c51 0x10ec7f 2024-09-16T16:45:45.424 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:45.531 INFO:teuthology.orchestra.run.smithi123.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: 0x82dff 0xbca834f 0xbb80cd9 0xbb82244 0xbb8245a 0xbb825b0 0xbb82a79 0x3e6ef 0x9e33944 0x9ea2e39 0x9ed3631 0x9e3abc4 0x9959157 0x9878562 0x989cc22 0x989cd81 0x98ae61e 0x98ae7ee 0x98aeafc 0x98aee0b 0x98aef6a 0x98aefe5 0x98af22c 0x98af4f0 0x98af580 0x98af624 0x95c34b2 0x95c3717 0x95cbcbc 0x96be366 0xbb95287 0xbbad105 0xbc5a9af 0xbc5bffd 0xb92a8cd 0xb92b105 0x3805438 0x2958f 0x2963f 0x35ec9b4 2024-09-16T16:45:45.531 INFO:teuthology.orchestra.run.smithi123.stderr:kernel callstack: 2024-09-16T16:45:46.500 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:46.501 INFO:teuthology.orchestra.run.smithi123.stderr:Suppressions used: 2024-09-16T16:45:46.501 INFO:teuthology.orchestra.run.smithi123.stderr: count bytes template 2024-09-16T16:45:46.501 INFO:teuthology.orchestra.run.smithi123.stderr: 1 8 MallocExtension::Register 2024-09-16T16:45:46.501 INFO:teuthology.orchestra.run.smithi123.stderr:----------------------------------------------------- 2024-09-16T16:45:46.501 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T16:45:46.524 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T16:45:46.549 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T16:45:46.549 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:46.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T16:45:46.614 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:46.614 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T16:45:46.679 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:46.679 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T16:45:46.743 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:46.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T16:45:46.807 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:46.807 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T16:45:46.862 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T16:45:46.862 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCuYOhmTO2GIBAArt0FoLm1Pzf0GJ/BqqyuZA== 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCyYOhmLlpACBAAb4WTdI5YFVLzlxfINj6aZw== 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQC1YOhmE+EcFBAAIkRurc9u+8bOoJpAOBGpUg== 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQC4YOhmGNMwHxAAPSumaxAMpgdHTb6W1J8gjA== 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2024-09-16T16:45:46.925 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCuYOhmNnCkIxAAMbAEF9gKcX7OmPZcht228g== 2024-09-16T16:45:46.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T16:45:47.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T16:45:47.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T16:45:47.148 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T16:45:47.209 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T16:45:47.272 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T16:45:47.272 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T16:45:47.297 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T16:45:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:47.430+0000 7fa0690ddc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:47.430+0000 7fa0690ddc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.444 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T16:45:47.469 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T16:45:47.524 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T16:45:47.524 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T16:45:47.525 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T16:45:47.566 INFO:tasks.ceph.mon.a:Started 2024-09-16T16:45:47.566 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T16:45:47.567 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T16:45:47.567 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T16:45:47.569 INFO:tasks.ceph.mgr.x:Started 2024-09-16T16:45:47.569 DEBUG:tasks.ceph:set 0 configs 2024-09-16T16:45:47.569 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T16:45:47.569 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T16:45:47.648 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-09-16T16:45:47.647+0000 7f834ea2fc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.648 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-09-16T16:45:47.647+0000 7f834ea2fc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.648 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-09-16T16:45:47.647+0000 7f834ea2fc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.648 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-16T16:45:47.648+0000 7fcad2435000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:47.672+0000 7f94ab791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:47.683+0000 7f94ab791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.753 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2024-09-16T16:45:47.764 INFO:tasks.ceph:check_enable_crimson: True 2024-09-16T16:45:47.764 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2024-09-16T16:45:47.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2024-09-16T16:45:47.853 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-16T16:45:47.853+0000 7fcad2435000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.853 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-16T16:45:47.853+0000 7fcad2435000 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.870 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:47.865+0000 7f26326bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:47.873+0000 7f26326bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:47.953 INFO:tasks.ceph.mon.a.smithi123.stderr:2024-09-16T16:45:47.953+0000 7f83453f4640 -1 WARNING: experimental feature 'crimson' is enabled 2024-09-16T16:45:47.954 INFO:tasks.ceph.mon.a.smithi123.stderr:Please be aware that this feature is experimental, untested, 2024-09-16T16:45:47.954 INFO:tasks.ceph.mon.a.smithi123.stderr:unsupported, and may result in data corruption, data loss, 2024-09-16T16:45:47.954 INFO:tasks.ceph.mon.a.smithi123.stderr:and/or irreparable damage to your cluster. Do not use 2024-09-16T16:45:47.954 INFO:tasks.ceph.mon.a.smithi123.stderr:feature with important data. 2024-09-16T16:45:47.954 INFO:tasks.ceph.mon.a.smithi123.stderr: 2024-09-16T16:45:48.765 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T16:45:48.766 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:48.766 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T16:45:48.790 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:48.790 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T16:45:48.857 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:45:48.857 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T16:45:48.922 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new 053aba5d-d94f-4e73-94b4-75158ee9a96b 0 2024-09-16T16:45:49.070 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.063+0000 7fd8b4474640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:49.074 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.074+0000 7fd8b4474640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:49.150 INFO:teuthology.orchestra.run.smithi123.stdout:0 2024-09-16T16:45:49.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new d2f0256b-4436-4985-82c9-69a8dbc8d9c8 1 2024-09-16T16:45:49.269 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.262+0000 7f3ff0a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:49.272 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.272+0000 7f3ff0a6c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:49.350 INFO:teuthology.orchestra.run.smithi123.stdout:1 2024-09-16T16:45:49.362 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd new c117682f-533a-498e-8b07-0dd88ef17d4a 2 2024-09-16T16:45:49.469 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.463+0000 7effb48a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:49.473 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.472+0000 7effb48a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:49.550 INFO:teuthology.orchestra.run.smithi123.stdout:2 2024-09-16T16:45:49.562 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2024-09-16T16:45:49.591 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T16:45:49.591 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2024-09-16T16:45:49.633 INFO:tasks.ceph.osd.0:Started 2024-09-16T16:45:49.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2024-09-16T16:45:49.662 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T16:45:49.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2024-09-16T16:45:49.705 INFO:tasks.ceph.osd.1:Started 2024-09-16T16:45:49.705 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2024-09-16T16:45:49.735 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T16:45:49.736 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2024-09-16T16:45:49.779 INFO:tasks.ceph.osd.2:Started 2024-09-16T16:45:49.779 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T16:45:49.953 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.945+0000 7efeff47d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:49.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:49.959+0000 7efeff47d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:50.048 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:45:50.048 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":6,"fsid":"0d2cd943-ff49-4927-aa46-a373e0b8c2c2","created":"2024-09-16T16:45:47.672610+0000","modified":"2024-09-16T16:45:49.550536+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T16:45:49.550536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"053aba5d-d94f-4e73-94b4-75158ee9a96b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d2f0256b-4436-4985-82c9-69a8dbc8d9c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c117682f-533a-498e-8b07-0dd88ef17d4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T16:45:50.060 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T16:45:50.060 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T16:45:50.362 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T16:45:50.466 INFO:teuthology.misc.health.smithi123.stderr:2024-09-16T16:45:50.462+0000 7fe8aaf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:50.471 INFO:teuthology.misc.health.smithi123.stderr:2024-09-16T16:45:50.471+0000 7fe8aaf91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:50.584 INFO:teuthology.misc.health.smithi123.stdout: 2024-09-16T16:45:50.584 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":6,"fsid":"0d2cd943-ff49-4927-aa46-a373e0b8c2c2","created":"2024-09-16T16:45:47.672610+0000","modified":"2024-09-16T16:45:49.550536+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T16:45:49.550536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"053aba5d-d94f-4e73-94b4-75158ee9a96b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d2f0256b-4436-4985-82c9-69a8dbc8d9c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c117682f-533a-498e-8b07-0dd88ef17d4a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T16:45:50.595 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-16T16:45:53.514 INFO:tasks.ceph.mgr.x.smithi123.stderr:2024-09-16T16:45:53.514+0000 7fca6f1d0640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-16T16:45:56.898 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T16:45:57.003 INFO:teuthology.misc.health.smithi123.stderr:2024-09-16T16:45:56.998+0000 7ff626791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:57.006 INFO:teuthology.misc.health.smithi123.stderr:2024-09-16T16:45:57.005+0000 7ff626791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:57.192 INFO:teuthology.misc.health.smithi123.stdout: 2024-09-16T16:45:57.193 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":10,"fsid":"0d2cd943-ff49-4927-aa46-a373e0b8c2c2","created":"2024-09-16T16:45:47.672610+0000","modified":"2024-09-16T16:45:56.579963+0000","last_up_change":"2024-09-16T16:45:55.570159+0000","last_in_change":"2024-09-16T16:45:49.550536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T16:45:56.535754+0000","flags":294945,"flags_names":"hashpspool,nopgchange,creating,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"053aba5d-d94f-4e73-94b4-75158ee9a96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1432985695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":1432985695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":1432985695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1432985695}]},"public_addr":"172.21.15.123:6802/1432985695","cluster_addr":"172.21.15.123:6803/1432985695","heartbeat_back_addr":"172.21.15.123:6811/1432985695","heartbeat_front_addr":"172.21.15.123:6812/1432985695","state":["exists","up"]},{"osd":1,"uuid":"d2f0256b-4436-4985-82c9-69a8dbc8d9c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":1330500426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1330500426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":1330500426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1330500426}]},"public_addr":"172.21.15.123:6809/1330500426","cluster_addr":"172.21.15.123:6808/1330500426","heartbeat_back_addr":"172.21.15.123:6805/1330500426","heartbeat_front_addr":"172.21.15.123:6806/1330500426","state":["exists","up"]},{"osd":2,"uuid":"c117682f-533a-498e-8b07-0dd88ef17d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1440730160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1440730160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":1440730160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1440730160}]},"public_addr":"172.21.15.123:6804/1440730160","cluster_addr":"172.21.15.123:6807/1440730160","heartbeat_back_addr":"172.21.15.123:6813/1440730160","heartbeat_front_addr":"172.21.15.123:6810/1440730160","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T16:45:57.204 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-16T16:45:57.204 INFO:tasks.ceph:Creating RBD pool 2024-09-16T16:45:57.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T16:45:57.312 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:57.303+0000 7f9b70d11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:57.315 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:57.314+0000 7f9b70d11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:57.584 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'rbd' created 2024-09-16T16:45:57.605 DEBUG:teuthology.orchestra.run.smithi123:> rbd --cluster ceph pool init rbd 2024-09-16T16:45:57.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:57.694+0000 7f6888610ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:57.695 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:57.694+0000 7f6888610ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:45:57.701 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:45:57.700+0000 7f6888610ac0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:00.659 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T16:46:00.660 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-16T16:46:00.660 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T16:46:00.770 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:00.764+0000 7fabec351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:00.773 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:00.772+0000 7fabec351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:00.957 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:46:00.971 INFO:teuthology.orchestra.run.smithi123.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2024-09-16T16:46:00.971 INFO:tasks.ceph_manager:config epoch is 1 2024-09-16T16:46:00.971 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T16:46:00.971 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T16:46:00.972 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T16:46:01.078 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:01.075+0000 7f0e1e591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:01.082 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:01.081+0000 7f0e1e591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:01.311 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:46:01.323 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":2034376298},{"type":"v1","addr":"172.21.15.123:6801","nonce":2034376298}]},"active_addr":"172.21.15.123:6801/2034376298","active_change":"2024-09-16T16:45:52.494765+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":567174526}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2550479201}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":2490564832}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3792068153}]}]} 2024-09-16T16:46:01.327 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T16:46:01.327 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T16:46:01.327 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T16:46:01.435 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:01.430+0000 7fc73c857640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:01.438 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:01.438+0000 7fc73c857640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:01.624 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:46:01.624 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":14,"fsid":"0d2cd943-ff49-4927-aa46-a373e0b8c2c2","created":"2024-09-16T16:45:47.672610+0000","modified":"2024-09-16T16:46:00.597475+0000","last_up_change":"2024-09-16T16:45:55.570159+0000","last_in_change":"2024-09-16T16:45:49.550536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T16:45:56.535754+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T16:45:57.501018+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"053aba5d-d94f-4e73-94b4-75158ee9a96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1432985695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":1432985695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":1432985695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1432985695}]},"public_addr":"172.21.15.123:6802/1432985695","cluster_addr":"172.21.15.123:6803/1432985695","heartbeat_back_addr":"172.21.15.123:6811/1432985695","heartbeat_front_addr":"172.21.15.123:6812/1432985695","state":["exists","up"]},{"osd":1,"uuid":"d2f0256b-4436-4985-82c9-69a8dbc8d9c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":1330500426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1330500426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":1330500426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1330500426}]},"public_addr":"172.21.15.123:6809/1330500426","cluster_addr":"172.21.15.123:6808/1330500426","heartbeat_back_addr":"172.21.15.123:6805/1330500426","heartbeat_front_addr":"172.21.15.123:6806/1330500426","state":["exists","up"]},{"osd":2,"uuid":"c117682f-533a-498e-8b07-0dd88ef17d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1440730160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1440730160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":1440730160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1440730160}]},"public_addr":"172.21.15.123:6804/1440730160","cluster_addr":"172.21.15.123:6807/1440730160","heartbeat_back_addr":"172.21.15.123:6813/1440730160","heartbeat_front_addr":"172.21.15.123:6810/1440730160","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T16:46:01.637 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T16:46:01.638 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T16:46:01.745 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:01.740+0000 7fd83ff91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:01.748 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:01.748+0000 7fd83ff91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:01.932 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:46:01.932 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":14,"fsid":"0d2cd943-ff49-4927-aa46-a373e0b8c2c2","created":"2024-09-16T16:45:47.672610+0000","modified":"2024-09-16T16:46:00.597475+0000","last_up_change":"2024-09-16T16:45:55.570159+0000","last_in_change":"2024-09-16T16:45:49.550536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T16:45:56.535754+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T16:45:57.501018+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"053aba5d-d94f-4e73-94b4-75158ee9a96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1432985695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":1432985695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":1432985695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1432985695}]},"public_addr":"172.21.15.123:6802/1432985695","cluster_addr":"172.21.15.123:6803/1432985695","heartbeat_back_addr":"172.21.15.123:6811/1432985695","heartbeat_front_addr":"172.21.15.123:6812/1432985695","state":["exists","up"]},{"osd":1,"uuid":"d2f0256b-4436-4985-82c9-69a8dbc8d9c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":1330500426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1330500426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":1330500426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1330500426}]},"public_addr":"172.21.15.123:6809/1330500426","cluster_addr":"172.21.15.123:6808/1330500426","heartbeat_back_addr":"172.21.15.123:6805/1330500426","heartbeat_front_addr":"172.21.15.123:6806/1330500426","state":["exists","up"]},{"osd":2,"uuid":"c117682f-533a-498e-8b07-0dd88ef17d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1440730160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1440730160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":1440730160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1440730160}]},"public_addr":"172.21.15.123:6804/1440730160","cluster_addr":"172.21.15.123:6807/1440730160","heartbeat_back_addr":"172.21.15.123:6813/1440730160","heartbeat_front_addr":"172.21.15.123:6810/1440730160","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T16:46:01.945 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T16:46:01.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T16:46:01.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T16:46:02.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.055+0000 7fc998591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.055+0000 7f3db4191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.060 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.055+0000 7f2458911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.064 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.064+0000 7f2458911640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.064+0000 7f3db4191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.065 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.064+0000 7fc998591640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.095 INFO:teuthology.orchestra.run.smithi123.stdout:34359738375 2024-09-16T16:46:02.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T16:46:02.099 INFO:teuthology.orchestra.run.smithi123.stdout:34359738375 2024-09-16T16:46:02.099 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T16:46:02.100 INFO:teuthology.orchestra.run.smithi123.stdout:38654705670 2024-09-16T16:46:02.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T16:46:02.207 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.202+0000 7f3dc4f08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.212 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.211+0000 7f3dc4f08640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.246+0000 7f733a791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.246+0000 7f100a151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.255+0000 7f100a151640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.255 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.255+0000 7f733a791640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.411 INFO:teuthology.orchestra.run.smithi123.stdout:34359738375 2024-09-16T16:46:02.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738375 for osd.1 2024-09-16T16:46:02.425 DEBUG:teuthology.parallel:result is None 2024-09-16T16:46:02.450 INFO:teuthology.orchestra.run.smithi123.stdout:38654705670 2024-09-16T16:46:02.456 INFO:teuthology.orchestra.run.smithi123.stdout:34359738375 2024-09-16T16:46:02.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2024-09-16T16:46:02.464 DEBUG:teuthology.parallel:result is None 2024-09-16T16:46:02.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738375 got 34359738375 for osd.0 2024-09-16T16:46:02.470 DEBUG:teuthology.parallel:result is None 2024-09-16T16:46:02.470 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T16:46:02.470 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T16:46:02.618 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.614+0000 7f63e2f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.621 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.621+0000 7f63e2f91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.804 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:46:02.804 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T16:46:02.816 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-16T16:46:02.505921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":3348,"kb_used_data":3348,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156889836,"statfs":{"total":160658620416,"available":160655192064,"internally_reserved":0,"allocated":3428352,"data_stored":3428352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.419688"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.627898+0000","last_change":"2024-09-16T16:46:00.635813+0000","last_active":"2024-09-16T16:46:00.627898+0000","last_peered":"2024-09-16T16:46:00.627898+0000","last_clean":"2024-09-16T16:46:00.627898+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T16:46:00.627898+0000","last_undegraded":"2024-09-16T16:46:00.627898+0000","last_fullsized":"2024-09-16T16:46:00.627898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.632614+0000","last_change":"2024-09-16T16:46:00.639079+0000","last_active":"2024-09-16T16:46:00.632614+0000","last_peered":"2024-09-16T16:46:00.632614+0000","last_clean":"2024-09-16T16:46:00.632614+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T16:46:00.632614+0000","last_undegraded":"2024-09-16T16:46:00.632614+0000","last_fullsized":"2024-09-16T16:46:00.632614+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.626621+0000","last_change":"2024-09-16T16:46:00.635095+0000","last_active":"2024-09-16T16:46:00.626621+0000","last_peered":"2024-09-16T16:46:00.626621+0000","last_clean":"2024-09-16T16:46:00.626621+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T16:46:00.626621+0000","last_undegraded":"2024-09-16T16:46:00.626621+0000","last_fullsized":"2024-09-16T16:46:00.626621+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.625913+0000","last_change":"2024-09-16T16:46:00.635505+0000","last_active":"2024-09-16T16:46:00.625913+0000","last_peered":"2024-09-16T16:46:00.625913+0000","last_clean":"2024-09-16T16:46:00.625913+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T16:46:00.625913+0000","last_undegraded":"2024-09-16T16:46:00.625913+0000","last_fullsized":"2024-09-16T16:46:00.625913+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":17,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.637335+0000","last_change":"2024-09-16T16:46:00.637335+0000","last_active":"2024-09-16T16:46:00.637335+0000","last_peered":"2024-09-16T16:46:00.637335+0000","last_clean":"2024-09-16T16:46:00.637335+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T16:46:00.637335+0000","last_undegraded":"2024-09-16T16:46:00.637335+0000","last_fullsized":"2024-09-16T16:46:00.637335+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.636208+0000","last_change":"2024-09-16T16:46:00.640092+0000","last_active":"2024-09-16T16:46:00.636208+0000","last_peered":"2024-09-16T16:46:00.636208+0000","last_clean":"2024-09-16T16:46:00.636208+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T16:46:00.636208+0000","last_undegraded":"2024-09-16T16:46:00.636208+0000","last_fullsized":"2024-09-16T16:46:00.636208+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.630105+0000","last_change":"2024-09-16T16:46:00.634304+0000","last_active":"2024-09-16T16:46:00.630105+0000","last_peered":"2024-09-16T16:46:00.630105+0000","last_clean":"2024-09-16T16:46:00.630105+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T16:46:00.630105+0000","last_undegraded":"2024-09-16T16:46:00.630105+0000","last_fullsized":"2024-09-16T16:46:00.630105+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.634560+0000","last_change":"2024-09-16T16:46:00.638883+0000","last_active":"2024-09-16T16:46:00.634560+0000","last_peered":"2024-09-16T16:46:00.634560+0000","last_clean":"2024-09-16T16:46:00.634560+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T16:46:00.634560+0000","last_undegraded":"2024-09-16T16:46:00.634560+0000","last_fullsized":"2024-09-16T16:46:00.634560+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":84,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.625084+0000","last_change":"2024-09-16T16:45:57.684651+0000","last_active":"2024-09-16T16:46:00.625084+0000","last_peered":"2024-09-16T16:46:00.625084+0000","last_clean":"2024-09-16T16:46:00.625084+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T16:46:00.625084+0000","last_undegraded":"2024-09-16T16:46:00.625084+0000","last_fullsized":"2024-09-16T16:46:00.625084+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160655192064,"internally_reserved":0,"allocated":3428352,"data_stored":3428352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":428,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297300,"statfs":{"total":53552873472,"available":53552435200,"internally_reserved":0,"allocated":438272,"data_stored":438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1588,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296140,"statfs":{"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1332,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296396,"statfs":{"total":53552873472,"available":53551509504,"internally_reserved":0,"allocated":1363968,"data_stored":1363968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551509504,"internally_reserved":0,"allocated":1363968,"data_stored":1363968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53552435200,"internally_reserved":0,"allocated":438272,"data_stored":438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T16:46:02.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T16:46:02.924 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.920+0000 7f3e338d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:02.927 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:02.927+0000 7f3e338d1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:03.110 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:46:03.110 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T16:46:03.123 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-09-16T16:46:02.505921+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":3348,"kb_used_data":3348,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156889836,"statfs":{"total":160658620416,"available":160655192064,"internally_reserved":0,"allocated":3428352,"data_stored":3428352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.419688"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.627898+0000","last_change":"2024-09-16T16:46:00.635813+0000","last_active":"2024-09-16T16:46:00.627898+0000","last_peered":"2024-09-16T16:46:00.627898+0000","last_clean":"2024-09-16T16:46:00.627898+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T16:46:00.627898+0000","last_undegraded":"2024-09-16T16:46:00.627898+0000","last_fullsized":"2024-09-16T16:46:00.627898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.632614+0000","last_change":"2024-09-16T16:46:00.639079+0000","last_active":"2024-09-16T16:46:00.632614+0000","last_peered":"2024-09-16T16:46:00.632614+0000","last_clean":"2024-09-16T16:46:00.632614+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T16:46:00.632614+0000","last_undegraded":"2024-09-16T16:46:00.632614+0000","last_fullsized":"2024-09-16T16:46:00.632614+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.626621+0000","last_change":"2024-09-16T16:46:00.635095+0000","last_active":"2024-09-16T16:46:00.626621+0000","last_peered":"2024-09-16T16:46:00.626621+0000","last_clean":"2024-09-16T16:46:00.626621+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T16:46:00.626621+0000","last_undegraded":"2024-09-16T16:46:00.626621+0000","last_fullsized":"2024-09-16T16:46:00.626621+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.625913+0000","last_change":"2024-09-16T16:46:00.635505+0000","last_active":"2024-09-16T16:46:00.625913+0000","last_peered":"2024-09-16T16:46:00.625913+0000","last_clean":"2024-09-16T16:46:00.625913+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T16:46:00.625913+0000","last_undegraded":"2024-09-16T16:46:00.625913+0000","last_fullsized":"2024-09-16T16:46:00.625913+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":17,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.637335+0000","last_change":"2024-09-16T16:46:00.637335+0000","last_active":"2024-09-16T16:46:00.637335+0000","last_peered":"2024-09-16T16:46:00.637335+0000","last_clean":"2024-09-16T16:46:00.637335+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T16:46:00.637335+0000","last_undegraded":"2024-09-16T16:46:00.637335+0000","last_fullsized":"2024-09-16T16:46:00.637335+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.636208+0000","last_change":"2024-09-16T16:46:00.640092+0000","last_active":"2024-09-16T16:46:00.636208+0000","last_peered":"2024-09-16T16:46:00.636208+0000","last_clean":"2024-09-16T16:46:00.636208+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T16:46:00.636208+0000","last_undegraded":"2024-09-16T16:46:00.636208+0000","last_fullsized":"2024-09-16T16:46:00.636208+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.630105+0000","last_change":"2024-09-16T16:46:00.634304+0000","last_active":"2024-09-16T16:46:00.630105+0000","last_peered":"2024-09-16T16:46:00.630105+0000","last_clean":"2024-09-16T16:46:00.630105+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T16:46:00.630105+0000","last_undegraded":"2024-09-16T16:46:00.630105+0000","last_fullsized":"2024-09-16T16:46:00.630105+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":16,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.634560+0000","last_change":"2024-09-16T16:46:00.638883+0000","last_active":"2024-09-16T16:46:00.634560+0000","last_peered":"2024-09-16T16:46:00.634560+0000","last_clean":"2024-09-16T16:46:00.634560+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T16:46:00.634560+0000","last_undegraded":"2024-09-16T16:46:00.634560+0000","last_fullsized":"2024-09-16T16:46:00.634560+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":84,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T16:46:00.625084+0000","last_change":"2024-09-16T16:45:57.684651+0000","last_active":"2024-09-16T16:46:00.625084+0000","last_peered":"2024-09-16T16:46:00.625084+0000","last_clean":"2024-09-16T16:46:00.625084+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T16:46:00.625084+0000","last_undegraded":"2024-09-16T16:46:00.625084+0000","last_fullsized":"2024-09-16T16:46:00.625084+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160655192064,"internally_reserved":0,"allocated":3428352,"data_stored":3428352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":428,"kb_used_data":428,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52297300,"statfs":{"total":53552873472,"available":53552435200,"internally_reserved":0,"allocated":438272,"data_stored":438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1588,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296140,"statfs":{"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":1332,"kb_used_data":1332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52296396,"statfs":{"total":53552873472,"available":53551509504,"internally_reserved":0,"allocated":1363968,"data_stored":1363968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53551509504,"internally_reserved":0,"allocated":1363968,"data_stored":1363968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53551247360,"internally_reserved":0,"allocated":1626112,"data_stored":1626112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53552435200,"internally_reserved":0,"allocated":438272,"data_stored":438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T16:46:03.124 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T16:46:03.124 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T16:46:03.124 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T16:46:03.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T16:46:03.229 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:03.226+0000 7f4a0b191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:03.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T16:46:03.232+0000 7f4a0b191640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T16:46:03.452 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T16:46:03.452 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T16:46:03.465 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T16:46:03.465 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T16:46:03.478 INFO:tasks.workunit:Pulling workunits from ref 0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-09-16T16:46:03.478 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T16:46:03.478 DEBUG:teuthology.orchestra.run.smithi123:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T16:46:03.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T16:46:03.494 INFO:teuthology.orchestra.run.smithi123.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-16T16:46:03.494 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-16T16:46:03.549 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-16T16:46:03.549 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-16T16:46:03.605 INFO:tasks.workunit:timeout=3h 2024-09-16T16:46:03.605 INFO:tasks.workunit:cleanup=True 2024-09-16T16:46:03.606 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 0325fd3c11a225c0fd9eb38fab88aad1370ad779 2024-09-16T16:46:03.662 INFO:tasks.workunit.client.0.smithi123.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T16:46:55.009 INFO:tasks.workunit.client.0.smithi123.stderr:Note: switching to '0325fd3c11a225c0fd9eb38fab88aad1370ad779'. 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr: git switch -c 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-16T16:46:55.010 INFO:tasks.workunit.client.0.smithi123.stderr:Or undo this operation with: 2024-09-16T16:46:55.011 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-16T16:46:55.011 INFO:tasks.workunit.client.0.smithi123.stderr: git switch - 2024-09-16T16:46:55.011 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-16T16:46:55.011 INFO:tasks.workunit.client.0.smithi123.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T16:46:55.011 INFO:tasks.workunit.client.0.smithi123.stderr: 2024-09-16T16:46:55.011 INFO:tasks.workunit.client.0.smithi123.stderr:HEAD is now at 0325fd3c11a Merge pull request #58530 from zdover23/wip-doc-2024-07-11-radosgw-s3-authentication 2024-09-16T16:46:55.013 DEBUG:teuthology.orchestra.run.smithi123:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T16:46:55.118 INFO:tasks.workunit.client.0.smithi123.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T16:46:55.120 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T16:46:55.120 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T16:46:55.769 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T16:46:55.837 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T16:46:55.877 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T16:46:55.879 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T16:46:55.879 INFO:tasks.workunit.client.0.smithi123.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T16:46:55.918 INFO:tasks.workunit.client.0.smithi123.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T16:46:55.921 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-16T16:46:55.921 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T16:46:55.936 INFO:tasks.workunit:Running workunits matching rbd/crimson/test_crimson_librbd.sh on client.0... 2024-09-16T16:46:55.937 INFO:tasks.workunit:Running workunit rbd/crimson/test_crimson_librbd.sh... 2024-09-16T16:46:55.937 DEBUG:teuthology.orchestra.run.smithi123:workunit test rbd/crimson/test_crimson_librbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0325fd3c11a225c0fd9eb38fab88aad1370ad779 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/crimson/test_crimson_librbd.sh 2024-09-16T16:46:56.042 INFO:tasks.workunit.client.0.smithi123.stdout:seed 29620 2024-09-16T16:46:56.043 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestLibRBD.* 2024-09-16T16:46:56.043 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 171 tests from 1 test suite. 2024-09-16T16:46:56.043 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T16:46:56.043 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 171 tests from TestLibRBD 2024-09-16T16:46:56.089 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CreateAndStat 2024-09-16T16:46:57.693 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:46:59.970 INFO:tasks.workunit.client.0.smithi123.stdout:image has size 2097152 and order 22 2024-09-16T16:46:59.970 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CreateAndStat (3880 ms) 2024-09-16T16:46:59.970 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CreateWithSameDataPool 2024-09-16T16:46:59.970 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:00.184 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CreateWithSameDataPool (214 ms) 2024-09-16T16:47:00.184 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CreateAndStatPP 2024-09-16T16:47:00.184 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:00.419 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CreateAndStatPP (235 ms) 2024-09-16T16:47:00.419 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.GetId 2024-09-16T16:47:00.419 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:00.745 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.GetId (326 ms) 2024-09-16T16:47:00.745 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.GetIdPP 2024-09-16T16:47:00.745 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:01.065 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.GetIdPP (320 ms) 2024-09-16T16:47:01.065 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.GetBlockNamePrefix 2024-09-16T16:47:01.065 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:01.276 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.GetBlockNamePrefix (210 ms) 2024-09-16T16:47:01.276 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.GetBlockNamePrefixPP 2024-09-16T16:47:01.276 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:01.483 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.GetBlockNamePrefixPP (208 ms) 2024-09-16T16:47:01.484 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestGetCreateTimestamp 2024-09-16T16:47:01.484 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:01.710 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestGetCreateTimestamp (227 ms) 2024-09-16T16:47:01.711 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.GetCreateTimestampPP 2024-09-16T16:47:01.711 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:01.922 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.GetCreateTimestampPP (211 ms) 2024-09-16T16:47:01.922 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.OpenAio 2024-09-16T16:47:01.922 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:02.141 INFO:tasks.workunit.client.0.smithi123.stdout:image has size 2097152 and order 22 2024-09-16T16:47:02.142 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.OpenAio (220 ms) 2024-09-16T16:47:02.142 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.OpenAioFail 2024-09-16T16:47:02.154 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.OpenAioFail (12 ms) 2024-09-16T16:47:02.154 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.OpenAioPP 2024-09-16T16:47:02.154 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:02.479 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.OpenAioPP (326 ms) 2024-09-16T16:47:02.480 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.OpenAioFailPP 2024-09-16T16:47:02.492 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.OpenAioFailPP (13 ms) 2024-09-16T16:47:02.492 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ResizeAndStat 2024-09-16T16:47:02.492 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:03.057 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ResizeAndStat (565 ms) 2024-09-16T16:47:03.057 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ResizeAndStatPP 2024-09-16T16:47:03.057 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:03.555 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ResizeAndStatPP (497 ms) 2024-09-16T16:47:03.555 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.UpdateWatchAndResize 2024-09-16T16:47:03.555 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:04.113 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.UpdateWatchAndResize (559 ms) 2024-09-16T16:47:04.114 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.UpdateWatchAndResizePP 2024-09-16T16:47:04.114 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:04.676 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.UpdateWatchAndResizePP (563 ms) 2024-09-16T16:47:04.677 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCreateLsDelete 2024-09-16T16:47:06.943 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:08.921 INFO:tasks.workunit.client.0.smithi123.stdout:image: image18 2024-09-16T16:47:08.921 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image18 2024-09-16T16:47:08.922 INFO:tasks.workunit.client.0.smithi123.stdout:found image18 2024-09-16T16:47:08.922 INFO:tasks.workunit.client.0.smithi123.stdout:erased image18 2024-09-16T16:47:08.922 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:09.974 INFO:tasks.workunit.client.0.smithi123.stdout:image: image18 2024-09-16T16:47:09.974 INFO:tasks.workunit.client.0.smithi123.stdout:image: image19 2024-09-16T16:47:09.974 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image18 2024-09-16T16:47:09.974 INFO:tasks.workunit.client.0.smithi123.stdout:found image18 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:erased image18 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image19 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:found image19 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:erased image19 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:image: image19 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image19 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:found image19 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:erased image19 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCreateLsDelete (5298 ms) 2024-09-16T16:47:09.975 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCreateLsDeletePP 2024-09-16T16:47:11.739 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:13.964 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 1 2024-09-16T16:47:13.964 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:47:13.964 INFO:tasks.workunit.client.0.smithi123.stdout:image: image20 2024-09-16T16:47:13.964 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image20 2024-09-16T16:47:13.964 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:14.094 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 2 2024-09-16T16:47:14.094 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T16:47:14.094 INFO:tasks.workunit.client.0.smithi123.stdout:image: image20 2024-09-16T16:47:14.094 INFO:tasks.workunit.client.0.smithi123.stdout:image: image21 2024-09-16T16:47:14.094 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image20 2024-09-16T16:47:14.094 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image21 2024-09-16T16:47:14.985 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 1 2024-09-16T16:47:14.985 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:47:14.985 INFO:tasks.workunit.client.0.smithi123.stdout:image: image21 2024-09-16T16:47:14.985 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image21 2024-09-16T16:47:14.985 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCreateLsDeletePP (5010 ms) 2024-09-16T16:47:14.985 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCopy 2024-09-16T16:47:16.760 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:image: image22 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image22 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:found image22 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:erased image22 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:image: image22 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:image: image23 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image22 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:found image22 2024-09-16T16:47:26.197 INFO:tasks.workunit.client.0.smithi123.stdout:erased image22 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image23 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:found image23 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:erased image23 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:image: image22 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:image: image23 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:image: image24 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image22 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:found image22 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:erased image22 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image23 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:found image23 2024-09-16T16:47:26.198 INFO:tasks.workunit.client.0.smithi123.stdout:erased image23 2024-09-16T16:47:26.199 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image24 2024-09-16T16:47:26.199 INFO:tasks.workunit.client.0.smithi123.stdout:found image24 2024-09-16T16:47:26.199 INFO:tasks.workunit.client.0.smithi123.stdout:erased image24 2024-09-16T16:47:26.199 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCopy (11212 ms) 2024-09-16T16:47:26.199 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCopyPP 2024-09-16T16:47:27.797 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:35.364 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 1 2024-09-16T16:47:35.364 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:47:35.365 INFO:tasks.workunit.client.0.smithi123.stdout:image: image25 2024-09-16T16:47:35.365 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image25 2024-09-16T16:47:35.638 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 2 2024-09-16T16:47:35.638 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T16:47:35.638 INFO:tasks.workunit.client.0.smithi123.stdout:image: image25 2024-09-16T16:47:35.638 INFO:tasks.workunit.client.0.smithi123.stdout:image: image26 2024-09-16T16:47:35.638 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image25 2024-09-16T16:47:35.638 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image26 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 3 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 3 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:image: image25 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:image: image26 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:image: image27 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image25 2024-09-16T16:47:36.547 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image26 2024-09-16T16:47:36.548 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image27 2024-09-16T16:47:37.230 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCopyPP (11034 ms) 2024-09-16T16:47:37.230 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestDeepCopy 2024-09-16T16:47:38.821 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:47:56.158 INFO:tasks.workunit.client.0.smithi123.stdout:image: image28 2024-09-16T16:47:56.158 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:found image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:erased image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:image: image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:image: image29 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:found image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:erased image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image29 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:found image29 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:erased image29 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:image: image28 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:image: image29 2024-09-16T16:47:56.159 INFO:tasks.workunit.client.0.smithi123.stdout:image: image30 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image28 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:found image28 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:erased image28 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image29 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:found image29 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:erased image29 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image30 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:found image30 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:erased image30 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:image: image28 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:image: image29 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:image: image30 2024-09-16T16:47:56.160 INFO:tasks.workunit.client.0.smithi123.stdout:image: image31 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image28 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:found image28 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:erased image28 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image29 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:found image29 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:erased image29 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image30 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:found image30 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:erased image30 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image31 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:found image31 2024-09-16T16:47:56.161 INFO:tasks.workunit.client.0.smithi123.stdout:erased image31 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:image: image28 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:image: image29 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:image: image30 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:image: image31 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:image: image32 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image28 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:found image28 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:erased image28 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image29 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:found image29 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:erased image29 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image30 2024-09-16T16:47:56.162 INFO:tasks.workunit.client.0.smithi123.stdout:found image30 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:erased image30 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image31 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:found image31 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:erased image31 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image32 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:found image32 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:erased image32 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:image: image28 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:image: image29 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:image: image30 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:image: image31 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:image: image32 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:image: image33 2024-09-16T16:47:56.163 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image28 2024-09-16T16:47:56.164 INFO:tasks.workunit.client.0.smithi123.stdout:found image28 2024-09-16T16:47:56.164 INFO:tasks.workunit.client.0.smithi123.stdout:erased image28 2024-09-16T16:47:56.164 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image29 2024-09-16T16:47:56.164 INFO:tasks.workunit.client.0.smithi123.stdout:found image29 2024-09-16T16:47:56.164 INFO:tasks.workunit.client.0.smithi123.stdout:erased image29 2024-09-16T16:47:56.164 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image30 2024-09-16T16:47:56.164 INFO:tasks.workunit.client.0.smithi123.stdout:found image30 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:erased image30 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image31 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:found image31 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:erased image31 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image32 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:found image32 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:erased image32 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image33 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:found image33 2024-09-16T16:47:56.165 INFO:tasks.workunit.client.0.smithi123.stdout:erased image33 2024-09-16T16:47:56.166 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestDeepCopy (18927 ms) 2024-09-16T16:47:56.166 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestDeepCopyPP 2024-09-16T16:47:57.862 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:05.529 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 1 2024-09-16T16:48:05.529 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:48:05.529 INFO:tasks.workunit.client.0.smithi123.stdout:image: image34 2024-09-16T16:48:05.529 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image34 2024-09-16T16:48:05.941 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 2 2024-09-16T16:48:05.941 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T16:48:05.941 INFO:tasks.workunit.client.0.smithi123.stdout:image: image34 2024-09-16T16:48:05.942 INFO:tasks.workunit.client.0.smithi123.stdout:image: image35 2024-09-16T16:48:05.942 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image34 2024-09-16T16:48:05.942 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image35 2024-09-16T16:48:07.031 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T16:48:07.031 INFO:tasks.workunit.client.0.smithi123.stdout:num images is: 3 2024-09-16T16:48:07.031 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 3 2024-09-16T16:48:07.031 INFO:tasks.workunit.client.0.smithi123.stdout:image: image34 2024-09-16T16:48:07.031 INFO:tasks.workunit.client.0.smithi123.stdout:image: image35 2024-09-16T16:48:07.031 INFO:tasks.workunit.client.0.smithi123.stdout:image: image36 2024-09-16T16:48:07.031 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image34 2024-09-16T16:48:07.032 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image35 2024-09-16T16:48:07.032 INFO:tasks.workunit.client.0.smithi123.stdout:expected = image36 2024-09-16T16:48:07.692 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestDeepCopyPP (11534 ms) 2024-09-16T16:48:07.692 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCreateLsDeleteSnap 2024-09-16T16:48:07.692 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap1 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:found snap1 with size 2097152 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 2 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap1 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap2 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:found snap1 with size 2097152 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:found snap2 with size 4194304 2024-09-16T16:48:12.203 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap2 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:found snap2 with size 4194304 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 0 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCreateLsDeleteSnap (4511 ms) 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestGetSnapShotTimeStamp 2024-09-16T16:48:12.204 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:14.236 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestGetSnapShotTimeStamp (2033 ms) 2024-09-16T16:48:14.236 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCreateLsDeleteSnapPP 2024-09-16T16:48:14.236 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:15.213 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T16:48:15.213 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:48:15.213 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap1 2024-09-16T16:48:15.213 INFO:tasks.workunit.client.0.smithi123.stdout:found snap1 with size 2097152 2024-09-16T16:48:16.280 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 2 2024-09-16T16:48:16.280 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T16:48:16.280 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap1 2024-09-16T16:48:16.280 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap2 2024-09-16T16:48:16.280 INFO:tasks.workunit.client.0.smithi123.stdout:found snap1 with size 2097152 2024-09-16T16:48:16.280 INFO:tasks.workunit.client.0.smithi123.stdout:found snap2 with size 4194304 2024-09-16T16:48:17.196 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T16:48:17.196 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:48:17.197 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap2 2024-09-16T16:48:17.197 INFO:tasks.workunit.client.0.smithi123.stdout:found snap2 with size 4194304 2024-09-16T16:48:18.221 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 0 2024-09-16T16:48:18.221 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T16:48:18.281 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCreateLsDeleteSnapPP (4045 ms) 2024-09-16T16:48:18.281 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestGetNameIdSnapPP 2024-09-16T16:48:18.281 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:24.273 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 0 2024-09-16T16:48:24.273 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T16:48:24.334 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestGetNameIdSnapPP (6053 ms) 2024-09-16T16:48:24.334 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCreateLsRenameSnapPP 2024-09-16T16:48:24.334 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:25.249 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T16:48:25.250 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:48:25.250 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap1 2024-09-16T16:48:25.250 INFO:tasks.workunit.client.0.smithi123.stdout:found snap1 with size 2097152 2024-09-16T16:48:26.295 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 2 2024-09-16T16:48:26.295 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T16:48:26.295 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap1 2024-09-16T16:48:26.295 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap2 2024-09-16T16:48:26.295 INFO:tasks.workunit.client.0.smithi123.stdout:found snap1 with size 2097152 2024-09-16T16:48:26.295 INFO:tasks.workunit.client.0.smithi123.stdout:found snap2 with size 4194304 2024-09-16T16:48:26.392 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 2 2024-09-16T16:48:26.392 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T16:48:26.392 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap1-rename 2024-09-16T16:48:26.392 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap2 2024-09-16T16:48:26.392 INFO:tasks.workunit.client.0.smithi123.stdout:found snap1-rename with size 2097152 2024-09-16T16:48:26.392 INFO:tasks.workunit.client.0.smithi123.stdout:found snap2 with size 4194304 2024-09-16T16:48:27.435 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T16:48:27.435 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T16:48:27.435 INFO:tasks.workunit.client.0.smithi123.stdout:snap: snap2-rename 2024-09-16T16:48:27.435 INFO:tasks.workunit.client.0.smithi123.stdout:found snap2-rename with size 4194304 2024-09-16T16:48:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 0 2024-09-16T16:48:28.210 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T16:48:28.265 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCreateLsRenameSnapPP (3931 ms) 2024-09-16T16:48:28.265 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ConcurrentCreatesUnvalidatedPool 2024-09-16T16:48:29.972 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:29.973 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:29.973 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:29.973 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:36.389 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ConcurrentCreatesUnvalidatedPool (8124 ms) 2024-09-16T16:48:36.389 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RemoveFullTry 2024-09-16T16:48:40.984 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:48:59.339 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RemoveFullTry (22950 ms) 2024-09-16T16:48:59.340 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RemoveFullTryDataPool 2024-09-16T16:49:06.025 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:49:28.708 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:49:30.039 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:49:31.204 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:49:32.379 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:49:33.490 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:49:34.947 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RemoveFullTryDataPool (35608 ms) 2024-09-16T16:49:34.947 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RemoveFullTryNamespace 2024-09-16T16:49:37.613 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:49:59.467 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RemoveFullTryNamespace (24520 ms) 2024-09-16T16:49:59.467 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RemoveFullTryNamespaceDataPool 2024-09-16T16:50:03.781 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:50:28.836 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:50:30.314 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:50:31.437 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:50:32.554 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:50:33.677 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:50:35.099 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RemoveFullTryNamespaceDataPool (35633 ms) 2024-09-16T16:50:35.099 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestIO 2024-09-16T16:50:35.099 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:51:05.793 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.794 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:51:05.795 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.796 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:51:05.797 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:51:05.798 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:05.799 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:discard: 512~512 = 512 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 1536~512 = 0 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.800 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.801 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.802 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.803 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.804 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.805 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.806 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.807 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:05.808 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:05.809 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:05.809 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:05.809 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:05.809 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.646 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.646 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.646 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.646 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.646 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.647 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.648 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.649 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.650 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.651 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.652 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.653 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.654 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:51:30.655 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestIO (55545 ms) 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestEncryptionLUKS1 2024-09-16T16:51:30.656 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.906 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.907 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:52:09.908 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.909 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.910 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:52:09.911 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:discard: 512~512 = 512 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 1536~512 = 0 2024-09-16T16:52:09.912 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:52:09.913 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:52:09.914 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.915 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.916 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:52:09.917 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.918 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.919 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.920 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:09.921 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:09.922 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:09.922 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:09.922 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:09.922 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:09.922 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.918 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.918 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.918 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.918 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.918 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.918 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.918 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.919 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.920 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.921 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.922 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.923 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.924 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.925 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.926 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:52:45.927 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 4 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestEncryptionLUKS1 (75273 ms) 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestEncryptionLUKS2 2024-09-16T16:52:45.928 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:53:35.475 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:53:35.476 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:53:35.477 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:53:35.478 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:53:35.479 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:53:35.480 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:53:35.481 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:discard: 512~512 = 512 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.482 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 1536~512 = 0 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.483 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.484 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.485 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.486 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:53:35.487 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:53:35.488 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:53:35.489 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.490 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:53:35.491 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:53:35.492 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.213 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.214 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.215 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.216 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.217 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.218 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.219 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.220 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:54:13.221 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 4 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:54:13.222 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestEncryptionLUKS2 (87295 ms) 2024-09-16T16:54:13.223 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCloneEncryption 2024-09-16T16:54:13.223 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:read: 4 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCloneEncryption (81322 ms) 2024-09-16T16:55:34.536 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.LUKS1UnderLUKS2WithoutResize 2024-09-16T16:55:34.537 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:55:52.875 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.LUKS1UnderLUKS2WithoutResize (18340 ms) 2024-09-16T16:55:52.876 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.LUKS2UnderLUKS1WithoutResize 2024-09-16T16:55:52.876 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:56:10.044 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.LUKS2UnderLUKS1WithoutResize (17169 ms) 2024-09-16T16:56:10.044 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EncryptionFormatNoData 2024-09-16T16:56:10.044 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:56:15.137 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.EncryptionFormatNoData (5093 ms) 2024-09-16T16:56:15.137 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EncryptionLoadBadSize 2024-09-16T16:56:15.137 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:56:21.766 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.EncryptionLoadBadSize (6629 ms) 2024-09-16T16:56:21.766 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EncryptionLoadBadStripePattern 2024-09-16T16:56:21.767 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_librbd.cc:2905: Skipped 2024-09-16T16:56:21.767 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T16:56:21.767 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.EncryptionLoadBadStripePattern (0 ms) 2024-09-16T16:56:21.767 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EncryptionLoadFormatMismatch 2024-09-16T16:56:21.767 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:58:47.057 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.EncryptionLoadFormatMismatch (145290 ms) 2024-09-16T16:58:47.057 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EncryptedResize 2024-09-16T16:58:47.057 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:59:01.617 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.EncryptedResize (14561 ms) 2024-09-16T16:59:01.618 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EncryptedFlattenSmallData 2024-09-16T16:59:01.618 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:59:17.376 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.EncryptedFlattenSmallData (15759 ms) 2024-09-16T16:59:17.376 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestIOWithIOHint 2024-09-16T16:59:17.376 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T16:59:48.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:59:48.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:59:48.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:59:48.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:59:48.485 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.486 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:59:48.487 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:59:48.488 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.489 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:59:48.490 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:discard: 512~512 = 512 2024-09-16T16:59:48.491 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 1536~512 = 0 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.492 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.493 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.494 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T16:59:48.495 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.496 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:59:48.497 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:59:48.498 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:59:48.499 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T16:59:48.500 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.155 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.155 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.155 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.155 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.155 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.155 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.155 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.156 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.157 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.158 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.159 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.160 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.161 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.162 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:00:15.163 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestIOWithIOHint (57778 ms) 2024-09-16T17:00:15.164 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestDataPoolIO 2024-09-16T17:00:16.935 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:20.061 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.062 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.063 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.064 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:discard: 512~512 = 512 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 1536~512 = 0 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.065 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:20.066 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:20.066 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:20.066 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestDataPoolIO (4907 ms) 2024-09-16T17:00:20.066 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteMismatch 2024-09-16T17:00:20.066 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:20.467 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteMismatch (406 ms) 2024-09-16T17:00:20.495 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteMismatch 2024-09-16T17:00:20.495 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:20.866 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteMismatch (399 ms) 2024-09-16T17:00:20.866 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteSuccess 2024-09-16T17:00:20.866 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:21.337 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteSuccess (471 ms) 2024-09-16T17:00:21.338 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteSuccess 2024-09-16T17:00:21.338 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:21.780 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteSuccess (443 ms) 2024-09-16T17:00:21.780 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitUnaligned 2024-09-16T17:00:21.780 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:22.686 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitUnaligned (906 ms) 2024-09-16T17:00:22.686 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnaligned 2024-09-16T17:00:22.686 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:23.459 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnaligned (773 ms) 2024-09-16T17:00:23.459 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteTooLarge 2024-09-16T17:00:23.459 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:24.137 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteTooLarge (678 ms) 2024-09-16T17:00:24.138 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteTooLarge 2024-09-16T17:00:24.138 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:25.274 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteTooLarge (1137 ms) 2024-09-16T17:00:25.275 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitSuccess 2024-09-16T17:00:25.275 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:29.854 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitSuccess (4580 ms) 2024-09-16T17:00:29.855 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccess 2024-09-16T17:00:29.855 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:35.468 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccess (5613 ms) 2024-09-16T17:00:35.468 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteVIovecLenDiffers 2024-09-16T17:00:35.468 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:35.888 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteVIovecLenDiffers (421 ms) 2024-09-16T17:00:35.888 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteVMismatch 2024-09-16T17:00:35.888 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:36.276 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteVMismatch (388 ms) 2024-09-16T17:00:36.276 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteVSuccess 2024-09-16T17:00:36.276 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:36.679 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteVSuccess (403 ms) 2024-09-16T17:00:36.679 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestScatterGatherIO 2024-09-16T17:00:36.679 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:37.073 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestScatterGatherIO (394 ms) 2024-09-16T17:00:37.073 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestEmptyDiscard 2024-09-16T17:00:37.073 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:37.559 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:37.559 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 0~1048576 = 0 2024-09-16T17:00:37.559 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:37.559 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 0~4194304 = 0 2024-09-16T17:00:37.560 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:37.560 INFO:tasks.workunit.client.0.smithi123.stdout:aio discard: 3145728~1048576 = 0 2024-09-16T17:00:37.560 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestEmptyDiscard (486 ms) 2024-09-16T17:00:37.560 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestFUA 2024-09-16T17:00:37.560 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.542 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.543 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.544 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:39.545 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:39.546 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestFUA (1983 ms) 2024-09-16T17:00:39.546 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestIOPP 2024-09-16T17:00:39.546 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:00:39.827 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.827 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.827 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.828 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:00:39.829 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:40.181 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:start compare and write 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:compare and wrote: 512 2024-09-16T17:00:40.182 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.183 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T17:00:40.183 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:40.203 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:40.203 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T17:00:40.203 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.203 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T17:00:40.203 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:40.227 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:40.227 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T17:00:40.227 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.227 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T17:00:40.227 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:40.257 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:40.257 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T17:00:40.257 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.257 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T17:00:40.257 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:40.308 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:40.308 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T17:00:40.308 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.309 INFO:tasks.workunit.client.0.smithi123.stdout:started aio compare and write 2024-09-16T17:00:40.309 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:00:40.354 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:00:40.354 INFO:tasks.workunit.client.0.smithi123.stdout:finished aio compare and write 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:40.355 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:40.363 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:40.363 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:00:40.363 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.363 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:40.363 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:40.370 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:40.370 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:00:40.370 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.370 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:40.370 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:40.376 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:40.376 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:00:40.376 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.376 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:40.376 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:40.382 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:40.382 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:00:40.382 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:00:40.382 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:00:40.382 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:00:40.383 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:00:40.383 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:00:40.383 INFO:tasks.workunit.client.0.smithi123.stdout:discard: 512~512 2024-09-16T17:00:40.383 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:06.501 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:01:06.501 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:01:06.501 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:01:06.501 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:01:06.501 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:01:06.501 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.502 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.503 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.504 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.505 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.506 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:06.507 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.508 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.508 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:06.508 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:06.508 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:06.818 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:06.818 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:06.819 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.819 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.819 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:06.819 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:06.819 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:06.981 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:06.981 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:06.982 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:07.777 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:07.777 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:07.777 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:07.777 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:07.777 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:07.777 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:07.777 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:08.229 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:08.229 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:08.229 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:08.229 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:08.229 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:08.229 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:08.230 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:09.062 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:09.062 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:09.062 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:09.063 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:09.063 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:09.063 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:09.063 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:09.796 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:09.797 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:09.797 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:09.797 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:09.797 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:11.270 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:11.270 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:11.270 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:11.271 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:11.271 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:11.271 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:11.271 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:12.194 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:12.194 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:12.194 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:12.194 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:12.195 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:12.195 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:12.195 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:13.883 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:13.883 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:13.883 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:13.883 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:13.883 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:13.884 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:13.884 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:15.031 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:15.031 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:15.032 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:17.051 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:17.051 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:17.052 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:17.052 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:17.052 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:17.052 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:17.052 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:18.643 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:18.643 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:18.643 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:18.643 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:18.643 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:18.644 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:18.644 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:20.939 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:20.939 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:20.939 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:20.939 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:20.939 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:20.939 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:20.939 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:22.558 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:22.559 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:22.559 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:22.559 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:22.559 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:25.424 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:25.425 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:25.425 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:25.425 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:25.425 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:25.425 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:25.425 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:27.428 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:27.428 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:27.428 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:27.428 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:27.428 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:27.429 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:27.429 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:30.439 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:30.440 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:30.440 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:30.440 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:30.440 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:30.440 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:30.440 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:32.612 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:01:32.613 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:01:32.613 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestIOPP (53070 ms) 2024-09-16T17:01:32.613 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteCompareTooSmallPP 2024-09-16T17:01:32.613 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:32.987 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteCompareTooSmallPP (375 ms) 2024-09-16T17:01:32.987 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteCompareTooSmallPP 2024-09-16T17:01:32.987 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:33.535 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteCompareTooSmallPP (548 ms) 2024-09-16T17:01:33.535 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteWriteTooSmallPP 2024-09-16T17:01:33.535 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:33.923 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteWriteTooSmallPP (388 ms) 2024-09-16T17:01:33.923 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteWriteTooSmallPP 2024-09-16T17:01:33.923 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:34.391 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteWriteTooSmallPP (468 ms) 2024-09-16T17:01:34.391 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteMismatchPP 2024-09-16T17:01:34.391 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:34.833 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteMismatchPP (442 ms) 2024-09-16T17:01:34.833 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteMismatchPP 2024-09-16T17:01:34.833 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:35.200 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:35.279 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteMismatchPP (446 ms) 2024-09-16T17:01:35.280 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteMismatchBufferlistGreaterLenPP 2024-09-16T17:01:35.280 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:35.695 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteMismatchBufferlistGreaterLenPP (416 ms) 2024-09-16T17:01:35.695 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteMismatchBufferlistGreaterLenPP 2024-09-16T17:01:35.695 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:36.035 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:36.106 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteMismatchBufferlistGreaterLenPP (411 ms) 2024-09-16T17:01:36.106 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteSuccessPP 2024-09-16T17:01:36.106 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:36.606 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteSuccessPP (500 ms) 2024-09-16T17:01:36.606 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteSuccessPP 2024-09-16T17:01:36.607 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:36.948 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:37.018 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteSuccessPP (412 ms) 2024-09-16T17:01:37.018 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteSuccessBufferlistGreaterLenPP 2024-09-16T17:01:37.018 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:37.532 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteSuccessBufferlistGreaterLenPP (513 ms) 2024-09-16T17:01:37.532 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteSuccessBufferlistGreaterLenPP 2024-09-16T17:01:37.532 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:37.926 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:38.005 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteSuccessBufferlistGreaterLenPP (473 ms) 2024-09-16T17:01:38.005 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitUnalignedPP 2024-09-16T17:01:38.005 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:38.814 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitUnalignedPP (809 ms) 2024-09-16T17:01:38.814 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnalignedPP 2024-09-16T17:01:38.814 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:39.325 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:39.716 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnalignedPP (902 ms) 2024-09-16T17:01:39.716 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteTooLargePP 2024-09-16T17:01:39.716 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:40.601 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteTooLargePP (885 ms) 2024-09-16T17:01:40.601 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteTooLargePP 2024-09-16T17:01:40.601 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:41.047 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:41.573 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteTooLargePP (971 ms) 2024-09-16T17:01:41.573 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitSuccessPP 2024-09-16T17:01:41.573 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:46.434 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitSuccessPP (4861 ms) 2024-09-16T17:01:46.434 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccessPP 2024-09-16T17:01:46.434 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:50.846 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:51.291 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteStripeUnitSuccessPP (4857 ms) 2024-09-16T17:01:51.291 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestIOPPWithIOHint 2024-09-16T17:01:51.291 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:01:51.726 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:01:51.727 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:01:51.727 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:01:51.727 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:01:51.727 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:01:51.727 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.727 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:01:51.727 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:51.745 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:51.745 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:01:51.746 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.746 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:01:51.746 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:51.764 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:51.764 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:01:51.765 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.765 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:01:51.765 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:51.784 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:51.784 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:01:51.784 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.784 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:01:51.784 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:51.803 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:51.803 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:01:51.803 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.803 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:01:51.803 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:01:51.815 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:01:51.816 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:01:51.816 INFO:tasks.workunit.client.0.smithi123.stdout:read: 512 2024-09-16T17:01:51.816 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.816 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:01:51.816 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:01:51.822 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:01:51.822 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:01:51.822 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.822 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:01:51.822 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:01:51.828 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:01:51.828 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:01:51.828 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.828 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:01:51.828 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:01:51.834 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:01:51.834 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:01:51.834 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.834 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:01:51.835 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:01:51.840 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:01:51.841 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:01:51.841 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:01:51.841 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:01:51.841 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:finished read 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 0 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.821 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 16384 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 49152 2024-09-16T17:02:17.822 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 65536 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 98304 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.823 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 114688 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.824 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 147456 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 163840 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.825 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 196608 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 212992 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:17.826 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:17.827 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:17.827 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.827 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.827 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:17.827 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:17.827 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:17.841 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:17.841 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:17.841 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:17.841 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:17.841 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:17.841 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:17.841 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:18.071 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:18.071 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:18.071 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:18.071 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:18.072 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:18.072 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:18.072 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:18.250 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:18.250 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:18.250 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:18.250 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:18.251 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:19.011 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:19.012 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:19.012 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:19.012 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:19.012 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:19.012 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:19.012 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:19.542 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:19.542 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:19.542 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:19.542 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:19.542 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:19.542 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:19.542 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:20.425 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:20.426 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:20.426 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:20.426 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:20.426 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:20.426 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:20.426 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:21.025 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:21.026 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:22.411 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:22.412 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:22.412 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:22.412 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:22.412 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:22.412 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:22.412 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:23.498 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:23.498 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:23.498 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:23.498 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:23.498 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:23.498 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:23.498 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:25.059 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:25.060 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:25.060 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:25.060 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:25.060 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:25.060 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:25.060 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:26.245 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:26.245 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:26.245 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:26.246 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:28.344 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:29.747 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:29.747 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:29.747 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:29.747 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:29.747 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:29.747 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:29.747 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:32.103 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:32.103 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:32.103 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:32.103 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:32.103 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:32.103 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:32.103 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:33.762 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:33.763 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:36.509 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:36.509 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:36.509 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:36.510 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:36.510 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:36.510 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:36.510 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:38.398 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:38.398 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:38.399 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:38.399 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:38.399 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:38.399 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:38.399 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:41.455 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:41.455 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:41.455 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:41.455 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:41.455 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:41.455 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:41.455 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:finished writesame 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:to verify the data 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:verified 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:started writesame 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:expected fail, finished writesame 2024-09-16T17:02:43.709 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestIOPPWithIOHint (52417 ms) 2024-09-16T17:02:43.710 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestIOToSnapshot 2024-09-16T17:02:43.710 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:02:46.375 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 0 2024-09-16T17:02:46.375 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T17:02:46.375 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:snap: orig 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:found orig with size 2097152 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:read: 80 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:write test data! 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 80 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 2 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:snap: orig 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:snap: written 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:found orig with size 2097152 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:found written with size 2097152 2024-09-16T17:02:46.376 INFO:tasks.workunit.client.0.smithi123.stdout:read: 80 2024-09-16T17:02:46.377 INFO:tasks.workunit.client.0.smithi123.stdout:read: 80 2024-09-16T17:02:46.377 INFO:tasks.workunit.client.0.smithi123.stdout:read: 80 2024-09-16T17:02:46.377 INFO:tasks.workunit.client.0.smithi123.stdout:write to snapshot returned -30 2024-09-16T17:02:46.377 INFO:tasks.workunit.client.0.smithi123.stdout:Read-only file system 2024-09-16T17:02:46.920 INFO:tasks.workunit.client.0.smithi123.stdout:read: 80 2024-09-16T17:02:46.920 INFO:tasks.workunit.client.0.smithi123.stdout:read: 80 2024-09-16T17:02:46.920 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 80 2024-09-16T17:02:46.920 INFO:tasks.workunit.client.0.smithi123.stdout:opening testimg@orig 2024-09-16T17:02:46.920 INFO:tasks.workunit.client.0.smithi123.stdout:read: 80 2024-09-16T17:02:46.920 INFO:tasks.workunit.client.0.smithi123.stdout:write to snapshot returned -30 2024-09-16T17:02:46.920 INFO:tasks.workunit.client.0.smithi123.stdout:Read-only file system 2024-09-16T17:02:49.264 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 2 2024-09-16T17:02:49.264 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T17:02:49.264 INFO:tasks.workunit.client.0.smithi123.stdout:snap: orig 2024-09-16T17:02:49.264 INFO:tasks.workunit.client.0.smithi123.stdout:snap: written 2024-09-16T17:02:49.264 INFO:tasks.workunit.client.0.smithi123.stdout:found orig with size 2097152 2024-09-16T17:02:49.264 INFO:tasks.workunit.client.0.smithi123.stdout:found written with size 2097152 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 1 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:snap: orig 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:found orig with size 2097152 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 0 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestIOToSnapshot (5555 ms) 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestSnapshotDeletedIo 2024-09-16T17:02:49.265 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:02:51.264 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestSnapshotDeletedIo (2001 ms) 2024-09-16T17:02:51.265 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestClone 2024-09-16T17:02:51.265 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:made parent image "parent" 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:parent has no parent info 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:made snapshot "parent@parent_snap" 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:can't unprotect an unprotected snap 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:can't protect a protected snap 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:made and opened clone "child" 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:read: 8 2024-09-16T17:03:03.261 INFO:tasks.workunit.client.0.smithi123.stdout:read: 8 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:read: 8 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:sizes and overlaps are good between parent and child 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:child image successfully cloned all image-meta pairs 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:sized down clone, changed overlap 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:parent info: size 4194304 obj_size 4194304 parent_pool 18446744073709551615 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:sized up clone, changed size but not overlap or parent's size 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:can't remove parent while child still exists 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:can't remove parent while still protected 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:removed parent snap after unprotecting 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestClone (11996 ms) 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestClone2 2024-09-16T17:03:03.262 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:12.432 INFO:tasks.workunit.client.0.smithi123.stdout:made parent image "parent" 2024-09-16T17:03:12.432 INFO:tasks.workunit.client.0.smithi123.stdout:parent has no parent info 2024-09-16T17:03:12.432 INFO:tasks.workunit.client.0.smithi123.stdout:made snapshot "parent@parent_snap" 2024-09-16T17:03:12.432 INFO:tasks.workunit.client.0.smithi123.stdout:made and opened clone "child" 2024-09-16T17:03:12.433 INFO:tasks.workunit.client.0.smithi123.stdout:child image successfully cloned all image-meta pairs 2024-09-16T17:03:12.433 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestClone2 (9172 ms) 2024-09-16T17:03:12.433 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ListChildren 2024-09-16T17:03:15.908 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:29.925 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.925 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.925 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.925 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.925 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.925 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 1 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:child: image125 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125/105fa8149e29 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.926 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 2 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:child: image125 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125/105fa8149e29 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.927 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.928 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 3 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 3 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:child: image125 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:child: image127 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125/105fa8149e29 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127/105f9dafa389 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.929 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 3 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 3 2024-09-16T17:03:29.930 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:child: image125 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:child: image127 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125/105fa8149e29 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127/105f9dafa389 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.931 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 4 2024-09-16T17:03:29.932 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 4 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:child: image125 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:child: image127 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:child: image128 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125/105fa8149e29 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127/105f9dafa389 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128/105f783a2335 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.933 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:29.934 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:29.935 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:29.935 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:29.935 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.935 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:29.935 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:29.935 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.499 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image125 2024-09-16T17:03:39.499 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 4 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 4 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:child: image125 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:child: image127 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:child: image128 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image125/105fa8149e29 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:39.500 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127/105f9dafa389 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128/105f783a2335 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:39.501 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image127 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 3 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 3 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:child: image127 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:child: image128 2024-09-16T17:03:39.502 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image127/105f9dafa389 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128/105f783a2335 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.503 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 2 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 2 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:child: image128 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-17/image128/105f783a2335 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:trying to find test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.504 INFO:tasks.workunit.client.0.smithi123.stdout:checking test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 1 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 1 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:child: image126 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-16/image126/105fb8659e0d 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 0 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ListChildren (27067 ms) 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ListChildrenTiered 2024-09-16T17:03:39.505 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_librbd.cc:6995: Skipped 2024-09-16T17:03:39.506 INFO:tasks.workunit.client.0.smithi123.stdout:Not supported by crimson yet. Skipped 2024-09-16T17:03:39.506 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.ListChildrenTiered (0 ms) 2024-09-16T17:03:39.506 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.LockingPP 2024-09-16T17:03:39.506 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:40.303 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.LockingPP (804 ms) 2024-09-16T17:03:40.303 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.FlushAio 2024-09-16T17:03:40.303 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:46.246 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.FlushAio (5942 ms) 2024-09-16T17:03:46.246 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.FlushAioPP 2024-09-16T17:03:46.246 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:50.185 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.FlushAioPP (3939 ms) 2024-09-16T17:03:50.185 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ZeroLengthWrite 2024-09-16T17:03:50.185 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:50.600 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ZeroLengthWrite (416 ms) 2024-09-16T17:03:50.600 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ZeroLengthDiscard 2024-09-16T17:03:50.600 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:51.089 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ZeroLengthDiscard (489 ms) 2024-09-16T17:03:51.089 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ZeroLengthRead 2024-09-16T17:03:51.089 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:51.360 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ZeroLengthRead (271 ms) 2024-09-16T17:03:51.361 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.LargeCacheRead 2024-09-16T17:03:51.361 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:51.886 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.LargeCacheRead (526 ms) 2024-09-16T17:03:51.886 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestPendingAio 2024-09-16T17:03:51.886 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:03:57.519 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestPendingAio (5633 ms) 2024-09-16T17:03:57.520 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CompareAndWriteCopyup 2024-09-16T17:03:57.520 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:00.452 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:03.703 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CompareAndWriteCopyup (6184 ms) 2024-09-16T17:04:03.703 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CompareAndWriteCopyupMismatch 2024-09-16T17:04:03.703 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:06.451 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:09.805 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CompareAndWriteCopyupMismatch (6102 ms) 2024-09-16T17:04:09.805 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.Flatten 2024-09-16T17:04:09.805 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:12.744 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.Flatten (2939 ms) 2024-09-16T17:04:12.744 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.Sparsify 2024-09-16T17:04:12.744 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:13.334 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 32768 2024-09-16T17:04:13.334 INFO:tasks.workunit.client.0.smithi123.stdout:read: 32768 2024-09-16T17:04:13.334 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.Sparsify (590 ms) 2024-09-16T17:04:13.334 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SparsifyPP 2024-09-16T17:04:13.334 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:13.942 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SparsifyPP (608 ms) 2024-09-16T17:04:13.942 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapshotLimit 2024-09-16T17:04:13.942 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:19.532 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapshotLimit (5590 ms) 2024-09-16T17:04:19.533 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapshotLimitPP 2024-09-16T17:04:19.533 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:24.482 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapshotLimitPP (4950 ms) 2024-09-16T17:04:24.482 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RebuildObjectMapViaLockOwner 2024-09-16T17:04:24.483 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:25.461 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:04:25.461 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RebuildObjectMapViaLockOwner (979 ms) 2024-09-16T17:04:25.461 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RenameViaLockOwner 2024-09-16T17:04:25.461 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:26.922 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RenameViaLockOwner (1461 ms) 2024-09-16T17:04:26.923 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapCreateViaLockOwner 2024-09-16T17:04:26.923 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:28.637 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapCreateViaLockOwner (1715 ms) 2024-09-16T17:04:28.637 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapRemoveViaLockOwner 2024-09-16T17:04:28.637 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:30.641 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapRemoveViaLockOwner (2004 ms) 2024-09-16T17:04:30.642 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.UpdateFeaturesViaLockOwner 2024-09-16T17:04:30.642 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:31.534 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.UpdateFeaturesViaLockOwner (893 ms) 2024-09-16T17:04:31.534 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EnableJournalingViaLockOwner 2024-09-16T17:04:31.534 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_librbd.cc:9736: Skipped 2024-09-16T17:04:31.535 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:04:31.535 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.EnableJournalingViaLockOwner (0 ms) 2024-09-16T17:04:31.535 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapRemove2 2024-09-16T17:04:31.535 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:35.678 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T17:04:35.678 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapRemove2 (4144 ms) 2024-09-16T17:04:35.678 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapRenameViaLockOwner 2024-09-16T17:04:35.678 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:36.792 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapRenameViaLockOwner (1114 ms) 2024-09-16T17:04:36.792 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapProtectViaLockOwner 2024-09-16T17:04:36.792 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:38.902 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapProtectViaLockOwner (2110 ms) 2024-09-16T17:04:38.902 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapUnprotectViaLockOwner 2024-09-16T17:04:38.902 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:41.050 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapUnprotectViaLockOwner (2148 ms) 2024-09-16T17:04:41.050 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.FlattenViaLockOwner 2024-09-16T17:04:41.050 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:44.631 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.FlattenViaLockOwner (3580 ms) 2024-09-16T17:04:44.631 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ResizeViaLockOwner 2024-09-16T17:04:44.631 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:45.318 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ResizeViaLockOwner (688 ms) 2024-09-16T17:04:45.318 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SparsifyViaLockOwner 2024-09-16T17:04:45.318 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:04:45.981 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SparsifyViaLockOwner (663 ms) 2024-09-16T17:04:45.981 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ObjectMapConsistentSnap 2024-09-16T17:04:45.981 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:11.793 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ObjectMapConsistentSnap (25812 ms) 2024-09-16T17:05:11.794 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.Metadata 2024-09-16T17:05:11.794 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:15.313 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.Metadata (3520 ms) 2024-09-16T17:05:15.313 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.MetadataPP 2024-09-16T17:05:15.313 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:19.090 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.MetadataPP (3776 ms) 2024-09-16T17:05:19.090 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.UpdateFeatures 2024-09-16T17:05:19.090 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:21.147 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.UpdateFeatures (2058 ms) 2024-09-16T17:05:21.147 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.FeaturesBitmaskString 2024-09-16T17:05:21.147 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.FeaturesBitmaskString (0 ms) 2024-09-16T17:05:21.148 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RebuildObjectMap 2024-09-16T17:05:21.148 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:23.728 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:05:23.728 INFO:tasks.workunit.client.0.smithi123.stdout:25.00% done 2024-09-16T17:05:23.728 INFO:tasks.workunit.client.0.smithi123.stdout:50.00% done 2024-09-16T17:05:23.728 INFO:tasks.workunit.client.0.smithi123.stdout:75.00% done 2024-09-16T17:05:23.728 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RebuildObjectMap (2581 ms) 2024-09-16T17:05:23.728 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.RebuildNewObjectMap 2024-09-16T17:05:24.296 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:05:24.296 INFO:tasks.workunit.client.0.smithi123.stdout:25.00% done 2024-09-16T17:05:24.296 INFO:tasks.workunit.client.0.smithi123.stdout:50.00% done 2024-09-16T17:05:24.296 INFO:tasks.workunit.client.0.smithi123.stdout:75.00% done 2024-09-16T17:05:24.296 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.RebuildNewObjectMap (567 ms) 2024-09-16T17:05:24.296 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CheckObjectMap 2024-09-16T17:05:24.296 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:26.178 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:05:26.178 INFO:tasks.workunit.client.0.smithi123.stdout:25.00% done 2024-09-16T17:05:26.178 INFO:tasks.workunit.client.0.smithi123.stdout:50.00% done 2024-09-16T17:05:26.178 INFO:tasks.workunit.client.0.smithi123.stdout:75.00% done 2024-09-16T17:05:26.178 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CheckObjectMap (1883 ms) 2024-09-16T17:05:26.178 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.BlockingAIO 2024-09-16T17:05:26.178 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:26.731 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.BlockingAIO (553 ms) 2024-09-16T17:05:26.731 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ExclusiveLockTransition 2024-09-16T17:05:26.731 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:31.152 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ExclusiveLockTransition (4421 ms) 2024-09-16T17:05:31.153 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ExclusiveLockReadTransition 2024-09-16T17:05:31.153 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_librbd.cc:10793: Skipped 2024-09-16T17:05:31.153 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:05:31.153 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.ExclusiveLockReadTransition (0 ms) 2024-09-16T17:05:31.153 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CacheMayCopyOnWrite 2024-09-16T17:05:31.153 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:33.787 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CacheMayCopyOnWrite (2634 ms) 2024-09-16T17:05:33.787 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.FlushEmptyOpsOnExternalSnapshot 2024-09-16T17:05:33.787 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:35.552 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.FlushEmptyOpsOnExternalSnapshot (1766 ms) 2024-09-16T17:05:35.552 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestImageOptions 2024-09-16T17:05:39.136 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:05:39.136 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T17:05:39.136 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestImageOptions (3584 ms) 2024-09-16T17:05:39.136 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestImageOptionsPP 2024-09-16T17:05:41.768 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:05:41.768 INFO:tasks.workunit.client.0.smithi123.stdout:100.00% done 2024-09-16T17:05:41.768 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestImageOptionsPP (2632 ms) 2024-09-16T17:05:41.768 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EventSocketPipe 2024-09-16T17:05:41.768 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.EventSocketPipe (0 ms) 2024-09-16T17:05:41.769 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.EventSocketEventfd 2024-09-16T17:05:41.769 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.EventSocketEventfd (0 ms) 2024-09-16T17:05:41.769 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ImagePollIO 2024-09-16T17:05:41.769 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:42.349 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:wrote: 512 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:05:42.350 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:05:42.351 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:started write 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:write completion cb called! 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 0 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:finished write 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:05:42.352 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:created completion 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:started read 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:read completion cb called! 2024-09-16T17:05:42.353 INFO:tasks.workunit.client.0.smithi123.stdout:return value is: 512 2024-09-16T17:05:42.354 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ImagePollIO (581 ms) 2024-09-16T17:05:42.354 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.Mirror 2024-09-16T17:05:42.479 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:42.628 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:46.984 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.Mirror (4635 ms) 2024-09-16T17:05:46.984 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.MirrorPeerAttributes 2024-09-16T17:05:47.145 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.MirrorPeerAttributes (161 ms) 2024-09-16T17:05:47.145 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.CreateWithMirrorEnabled 2024-09-16T17:05:55.592 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.CreateWithMirrorEnabled (8447 ms) 2024-09-16T17:05:55.592 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.FlushCacheWithCopyupOnExternalSnapshot 2024-09-16T17:05:55.592 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:05:59.422 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.FlushCacheWithCopyupOnExternalSnapshot (3830 ms) 2024-09-16T17:05:59.422 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ExclusiveLock 2024-09-16T17:05:59.423 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:00.226 INFO:tasks.workunit.client.0.smithi123.stdout:1: acquiring exclusive lock 2024-09-16T17:06:00.226 INFO:tasks.workunit.client.0.smithi123.stdout:2: acquiring exclusive lock 2024-09-16T17:06:00.326 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock acquired 2024-09-16T17:06:00.327 INFO:tasks.workunit.client.0.smithi123.stdout:2: releasing exclusive lock 2024-09-16T17:06:00.458 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock released 2024-09-16T17:06:00.458 INFO:tasks.workunit.client.0.smithi123.stdout:2: acquiring exclusive lock 2024-09-16T17:06:00.586 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock acquired 2024-09-16T17:06:00.614 INFO:tasks.workunit.client.0.smithi123.stdout:2: releasing exclusive lock 2024-09-16T17:06:00.682 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock released 2024-09-16T17:06:00.682 INFO:tasks.workunit.client.0.smithi123.stdout:2: acquiring exclusive lock 2024-09-16T17:06:00.760 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock acquired 2024-09-16T17:06:00.765 INFO:tasks.workunit.client.0.smithi123.stdout:1: releasing exclusive lock 2024-09-16T17:06:00.835 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock released 2024-09-16T17:06:00.835 INFO:tasks.workunit.client.0.smithi123.stdout:1: acquiring exclusive lock 2024-09-16T17:06:00.931 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock acquired 2024-09-16T17:06:00.955 INFO:tasks.workunit.client.0.smithi123.stdout:2: releasing exclusive lock 2024-09-16T17:06:01.015 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock released 2024-09-16T17:06:01.015 INFO:tasks.workunit.client.0.smithi123.stdout:2: acquiring exclusive lock 2024-09-16T17:06:01.100 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock acquired 2024-09-16T17:06:01.111 INFO:tasks.workunit.client.0.smithi123.stdout:1: releasing exclusive lock 2024-09-16T17:06:01.173 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock released 2024-09-16T17:06:01.173 INFO:tasks.workunit.client.0.smithi123.stdout:1: acquiring exclusive lock 2024-09-16T17:06:01.264 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock acquired 2024-09-16T17:06:01.297 INFO:tasks.workunit.client.0.smithi123.stdout:2: releasing exclusive lock 2024-09-16T17:06:01.405 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock released 2024-09-16T17:06:01.405 INFO:tasks.workunit.client.0.smithi123.stdout:2: acquiring exclusive lock 2024-09-16T17:06:01.554 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock acquired 2024-09-16T17:06:01.591 INFO:tasks.workunit.client.0.smithi123.stdout:1: releasing exclusive lock 2024-09-16T17:06:01.752 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock released 2024-09-16T17:06:01.752 INFO:tasks.workunit.client.0.smithi123.stdout:1: acquiring exclusive lock 2024-09-16T17:06:01.839 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock acquired 2024-09-16T17:06:01.870 INFO:tasks.workunit.client.0.smithi123.stdout:2: releasing exclusive lock 2024-09-16T17:06:01.927 INFO:tasks.workunit.client.0.smithi123.stdout:2: exclusive lock released 2024-09-16T17:06:01.991 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock acquired 2024-09-16T17:06:01.999 INFO:tasks.workunit.client.0.smithi123.stdout:1: releasing exclusive lock 2024-09-16T17:06:02.055 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock released 2024-09-16T17:06:02.055 INFO:tasks.workunit.client.0.smithi123.stdout:1: acquiring exclusive lock 2024-09-16T17:06:02.122 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock acquired 2024-09-16T17:06:02.153 INFO:tasks.workunit.client.0.smithi123.stdout:1: releasing exclusive lock 2024-09-16T17:06:02.206 INFO:tasks.workunit.client.0.smithi123.stdout:1: exclusive lock released 2024-09-16T17:06:02.547 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ExclusiveLock (3124 ms) 2024-09-16T17:06:02.547 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.BreakLock 2024-09-16T17:06:02.547 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_librbd.cc:11553: Skipped 2024-09-16T17:06:02.547 INFO:tasks.workunit.client.0.smithi123.stdout:Not supported by crimson yet. Skipped 2024-09-16T17:06:02.547 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.BreakLock (1 ms) 2024-09-16T17:06:02.547 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.DiscardAfterWrite 2024-09-16T17:06:02.547 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:03.491 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.DiscardAfterWrite (943 ms) 2024-09-16T17:06:03.491 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.DefaultFeatures 2024-09-16T17:06:03.491 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.DefaultFeatures (1 ms) 2024-09-16T17:06:03.491 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestTrashMoveAndPurge 2024-09-16T17:06:03.491 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:1.56% done 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:3.12% done 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:4.69% done 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:6.25% done 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:7.81% done 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:9.38% done 2024-09-16T17:06:04.531 INFO:tasks.workunit.client.0.smithi123.stdout:10.94% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:12.50% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:14.06% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:15.62% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:17.19% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:18.75% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:20.31% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:21.88% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:23.44% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:25.00% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:26.56% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:28.12% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:29.69% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:31.25% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:32.81% done 2024-09-16T17:06:04.532 INFO:tasks.workunit.client.0.smithi123.stdout:34.38% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:35.94% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:37.50% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:39.06% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:40.62% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:42.19% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:43.75% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:45.31% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:46.88% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:48.44% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:50.00% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:51.56% done 2024-09-16T17:06:04.533 INFO:tasks.workunit.client.0.smithi123.stdout:53.12% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:54.69% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:56.25% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:57.81% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:59.38% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:60.94% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:62.50% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:64.06% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:65.62% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:67.19% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:68.75% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:70.31% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:71.88% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:73.44% done 2024-09-16T17:06:04.534 INFO:tasks.workunit.client.0.smithi123.stdout:75.00% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:76.56% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:78.12% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:79.69% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:81.25% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:82.81% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:84.38% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:85.94% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:87.50% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:89.06% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:90.62% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:92.19% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:93.75% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:95.31% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:96.88% done 2024-09-16T17:06:04.535 INFO:tasks.workunit.client.0.smithi123.stdout:98.44% done 2024-09-16T17:06:04.536 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestTrashMoveAndPurge (1040 ms) 2024-09-16T17:06:04.536 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestTrashMoveAndPurgeNonExpiredDelay 2024-09-16T17:06:04.536 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:05.623 INFO:tasks.workunit.client.0.smithi123.stdout:0.00% done 2024-09-16T17:06:05.623 INFO:tasks.workunit.client.0.smithi123.stdout:1.56% done 2024-09-16T17:06:05.623 INFO:tasks.workunit.client.0.smithi123.stdout:3.12% done 2024-09-16T17:06:05.623 INFO:tasks.workunit.client.0.smithi123.stdout:4.69% done 2024-09-16T17:06:05.623 INFO:tasks.workunit.client.0.smithi123.stdout:6.25% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:7.81% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:9.38% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:10.94% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:12.50% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:14.06% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:15.62% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:17.19% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:18.75% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:20.31% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:21.88% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:23.44% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:25.00% done 2024-09-16T17:06:05.624 INFO:tasks.workunit.client.0.smithi123.stdout:26.56% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:28.12% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:29.69% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:31.25% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:32.81% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:34.38% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:35.94% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:37.50% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:39.06% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:40.62% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:42.19% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:43.75% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:45.31% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:46.88% done 2024-09-16T17:06:05.625 INFO:tasks.workunit.client.0.smithi123.stdout:48.44% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:50.00% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:51.56% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:53.12% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:54.69% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:56.25% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:57.81% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:59.38% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:60.94% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:62.50% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:64.06% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:65.62% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:67.19% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:68.75% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:70.31% done 2024-09-16T17:06:05.626 INFO:tasks.workunit.client.0.smithi123.stdout:71.88% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:73.44% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:75.00% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:76.56% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:78.12% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:79.69% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:81.25% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:82.81% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:84.38% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:85.94% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:87.50% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:89.06% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:90.62% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:92.19% done 2024-09-16T17:06:05.627 INFO:tasks.workunit.client.0.smithi123.stdout:93.75% done 2024-09-16T17:06:05.628 INFO:tasks.workunit.client.0.smithi123.stdout:95.31% done 2024-09-16T17:06:05.628 INFO:tasks.workunit.client.0.smithi123.stdout:96.88% done 2024-09-16T17:06:05.628 INFO:tasks.workunit.client.0.smithi123.stdout:98.44% done 2024-09-16T17:06:05.628 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestTrashMoveAndPurgeNonExpiredDelay (1092 ms) 2024-09-16T17:06:05.628 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestTrashPurge 2024-09-16T17:06:05.628 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:05.742 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:08.014 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestTrashPurge (2391 ms) 2024-09-16T17:06:08.015 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestTrashMoveAndRestore 2024-09-16T17:06:08.015 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:08.829 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestTrashMoveAndRestore (814 ms) 2024-09-16T17:06:08.829 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestListWatchers 2024-09-16T17:06:08.829 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:09.333 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestListWatchers (505 ms) 2024-09-16T17:06:09.333 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestSetSnapById 2024-09-16T17:06:09.333 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:10.676 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestSetSnapById (1343 ms) 2024-09-16T17:06:10.676 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.Namespaces 2024-09-16T17:06:11.124 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.Namespaces (448 ms) 2024-09-16T17:06:11.124 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.NamespacesPP 2024-09-16T17:06:11.683 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:11.736 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:13.028 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.NamespacesPP (1904 ms) 2024-09-16T17:06:13.028 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.Migration 2024-09-16T17:06:13.028 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:13.028 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:20.915 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.Migration (7887 ms) 2024-09-16T17:06:20.916 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.MigrationPP 2024-09-16T17:06:20.916 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:20.916 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:28.656 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.MigrationPP (7741 ms) 2024-09-16T17:06:28.656 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestGetAccessTimestamp 2024-09-16T17:06:28.656 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:28.898 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestGetAccessTimestamp (242 ms) 2024-09-16T17:06:28.898 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.TestGetModifyTimestamp 2024-09-16T17:06:28.898 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:29.133 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.TestGetModifyTimestamp (235 ms) 2024-09-16T17:06:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ZeroOverlapFlatten 2024-09-16T17:06:29.134 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:31.803 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ZeroOverlapFlatten (2670 ms) 2024-09-16T17:06:31.803 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.PoolMetadata 2024-09-16T17:06:32.139 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.PoolMetadata (336 ms) 2024-09-16T17:06:32.140 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.PoolMetadataPP 2024-09-16T17:06:32.405 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.PoolMetadataPP (266 ms) 2024-09-16T17:06:32.406 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.Config 2024-09-16T17:06:32.467 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:33.082 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.Config (676 ms) 2024-09-16T17:06:33.082 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ConfigPP 2024-09-16T17:06:33.108 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:33.915 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ConfigPP (833 ms) 2024-09-16T17:06:33.915 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.PoolStatsPP 2024-09-16T17:06:35.311 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:38.991 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:39.098 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:39.197 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:41.525 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.PoolStatsPP (7610 ms) 2024-09-16T17:06:41.525 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ImageSpec 2024-09-16T17:06:43.343 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:06:51.345 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.ImageSpec (9820 ms) 2024-09-16T17:06:51.345 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.SnapRemoveWithChildMissing 2024-09-16T17:06:52.849 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:07:11.647 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 3 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 3 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout:child: image224 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout:child: image226 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout:child: image225 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-1/image224/105fd478d4cc 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-20/image225/105f7487265 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout:found child test-librbd-smithi123-29620-1/image226/105fdb078e28 2024-09-16T17:07:11.648 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:07:11.649 INFO:tasks.workunit.client.0.smithi123.stdout:num children is: 0 2024-09-16T17:07:11.649 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T17:07:11.649 INFO:tasks.workunit.client.0.smithi123.stdout:num snaps is: 0 2024-09-16T17:07:11.649 INFO:tasks.workunit.client.0.smithi123.stdout:expected: 0 2024-09-16T17:07:11.649 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.SnapRemoveWithChildMissing (20302 ms) 2024-09-16T17:07:11.649 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.QuiesceWatch 2024-09-16T17:07:13.004 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:07:25.780 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.QuiesceWatch (14133 ms) 2024-09-16T17:07:25.781 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.QuiesceWatchPP 2024-09-16T17:07:25.781 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:07:33.940 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.QuiesceWatchPP (8159 ms) 2024-09-16T17:07:33.940 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.QuiesceWatchError 2024-09-16T17:07:33.940 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_librbd.cc:13115: Skipped 2024-09-16T17:07:33.940 INFO:tasks.workunit.client.0.smithi123.stdout:Not supported by crimson yet. Skipped 2024-09-16T17:07:33.940 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.QuiesceWatchError (0 ms) 2024-09-16T17:07:33.940 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.QuiesceWatchTimeout 2024-09-16T17:07:33.940 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:07:34.163 INFO:tasks.workunit.client.0.smithi123.stderr:test quiesce is not long enough to time out 2024-09-16T17:07:43.537 INFO:tasks.workunit.client.0.smithi123.stderr:test quiesce is timed out 2024-09-16T17:07:43.611 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 0 2024-09-16T17:07:44.611 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 1 2024-09-16T17:07:45.611 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 2 2024-09-16T17:07:46.611 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 3 2024-09-16T17:07:47.611 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 4 2024-09-16T17:07:48.611 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 5 2024-09-16T17:07:49.612 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 6 2024-09-16T17:07:50.612 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 7 2024-09-16T17:07:51.612 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 8 2024-09-16T17:07:52.612 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 9 2024-09-16T17:07:53.612 INFO:tasks.workunit.client.0.smithi123.stderr:waiting for timed out ... 10 2024-09-16T17:08:04.612 INFO:tasks.workunit.client.0.smithi123.stderr:test retry succeeds 2024-09-16T17:08:08.971 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.QuiesceWatchTimeout (35031 ms) 2024-09-16T17:08:08.971 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.WriteZeroes 2024-09-16T17:08:08.971 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:08:09.441 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.WriteZeroes (470 ms) 2024-09-16T17:08:09.441 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.WriteZeroesThickProvision 2024-09-16T17:08:09.441 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:08:10.216 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.WriteZeroesThickProvision (775 ms) 2024-09-16T17:08:10.216 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.ConcurrentOperations 2024-09-16T17:08:10.216 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_librbd.cc:13474: Skipped 2024-09-16T17:08:10.216 INFO:tasks.workunit.client.0.smithi123.stdout:Not supported by crimson yet. Skipped 2024-09-16T17:08:10.216 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.ConcurrentOperations (0 ms) 2024-09-16T17:08:10.216 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestLibRBD.FormatAndCloneFormatOptions 2024-09-16T17:08:19.982 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestLibRBD.FormatAndCloneFormatOptions (9766 ms) 2024-09-16T17:08:39.907 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 171 tests from TestLibRBD (1283892 ms total) 2024-09-16T17:08:39.907 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:08:39.907 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:08:39.907 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 171 tests from 1 test suite ran. (1303864 ms total) 2024-09-16T17:08:39.907 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 164 tests. 2024-09-16T17:08:39.907 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] 7 tests, listed below: 2024-09-16T17:08:39.907 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.EncryptionLoadBadStripePattern 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.ListChildrenTiered 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.EnableJournalingViaLockOwner 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.ExclusiveLockReadTransition 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.BreakLock 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.QuiesceWatchError 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestLibRBD.ConcurrentOperations 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout: YOU HAVE 1 DISABLED TEST 2024-09-16T17:08:39.908 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:08:40.025 INFO:tasks.workunit.client.0.smithi123.stdout:seed 39320 2024-09-16T17:08:40.025 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/0.* 2024-09-16T17:08:40.025 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:08:40.025 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:08:40.025 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/0, where TypeParam = PlaintextUnderLUKS1 2024-09-16T17:08:40.098 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/0.Simple 2024-09-16T17:08:41.909 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:08:57.365 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/0.Simple (17267 ms) 2024-09-16T17:08:57.365 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/0.Grow 2024-09-16T17:08:57.365 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:09:11.481 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/0.Grow (14116 ms) 2024-09-16T17:09:11.481 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/0.Shrink 2024-09-16T17:09:11.482 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:09:25.464 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/0.Shrink (13983 ms) 2024-09-16T17:09:25.465 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/0.ShrinkToOne 2024-09-16T17:09:25.465 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:09:37.692 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/0.ShrinkToOne (12227 ms) 2024-09-16T17:09:37.692 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/0.ShrinkToOneAfterSnapshot 2024-09-16T17:09:37.692 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:09:59.188 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/0.ShrinkToOneAfterSnapshot (21497 ms) 2024-09-16T17:09:59.189 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/0.MinOverlap 2024-09-16T17:09:59.189 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:10:11.819 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/0.MinOverlap (12630 ms) 2024-09-16T17:10:11.819 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/0.ZeroOverlap 2024-09-16T17:10:11.819 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:10:22.994 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/0.ZeroOverlap (11176 ms) 2024-09-16T17:10:23.901 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/0 (102896 ms total) 2024-09-16T17:10:23.901 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:10:23.901 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:10:23.902 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (103876 ms total) 2024-09-16T17:10:23.902 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:10:23.996 INFO:tasks.workunit.client.0.smithi123.stdout:seed 39391 2024-09-16T17:10:23.996 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/1.* 2024-09-16T17:10:23.996 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:10:23.996 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:10:23.997 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/1, where TypeParam = PlaintextUnderLUKS2 2024-09-16T17:10:24.096 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/1.Simple 2024-09-16T17:10:25.906 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:10:49.467 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/1.Simple (25370 ms) 2024-09-16T17:10:49.467 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/1.Grow 2024-09-16T17:10:49.467 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:11:09.623 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/1.Grow (20157 ms) 2024-09-16T17:11:09.624 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/1.Shrink 2024-09-16T17:11:09.624 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:11:29.404 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/1.Shrink (19780 ms) 2024-09-16T17:11:29.404 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/1.ShrinkToOne 2024-09-16T17:11:29.404 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:11:45.523 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/1.ShrinkToOne (16119 ms) 2024-09-16T17:11:45.523 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/1.ShrinkToOneAfterSnapshot 2024-09-16T17:11:45.523 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:12:08.165 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/1.ShrinkToOneAfterSnapshot (22642 ms) 2024-09-16T17:12:08.166 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/1.MinOverlap 2024-09-16T17:12:08.166 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:12:23.103 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/1.MinOverlap (14938 ms) 2024-09-16T17:12:23.103 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/1.ZeroOverlap 2024-09-16T17:12:23.103 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:12:37.064 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/1.ZeroOverlap (13961 ms) 2024-09-16T17:12:38.045 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/1 (132968 ms total) 2024-09-16T17:12:38.045 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:12:38.045 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:12:38.045 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (134048 ms total) 2024-09-16T17:12:38.045 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:12:38.110 INFO:tasks.workunit.client.0.smithi123.stdout:seed 45955 2024-09-16T17:12:38.110 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/2.* 2024-09-16T17:12:38.111 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:12:38.111 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:12:38.111 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/2, where TypeParam = UnformattedLUKS1 2024-09-16T17:12:38.202 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/2.Simple 2024-09-16T17:12:40.050 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:13:02.796 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/2.Simple (24594 ms) 2024-09-16T17:13:02.797 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/2.Grow 2024-09-16T17:13:02.797 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:13:19.335 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/2.Grow (16538 ms) 2024-09-16T17:13:19.335 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/2.Shrink 2024-09-16T17:13:19.335 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:13:36.321 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/2.Shrink (16987 ms) 2024-09-16T17:13:36.322 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/2.ShrinkToOne 2024-09-16T17:13:36.322 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:13:51.016 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/2.ShrinkToOne (14695 ms) 2024-09-16T17:13:51.016 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/2.ShrinkToOneAfterSnapshot 2024-09-16T17:13:51.016 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:14:08.961 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/2.ShrinkToOneAfterSnapshot (17945 ms) 2024-09-16T17:14:08.961 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/2.MinOverlap 2024-09-16T17:14:08.961 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:14:23.629 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/2.MinOverlap (14668 ms) 2024-09-16T17:14:23.629 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/2.ZeroOverlap 2024-09-16T17:14:23.629 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:14:36.294 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/2.ZeroOverlap (12665 ms) 2024-09-16T17:14:37.067 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/2 (118092 ms total) 2024-09-16T17:14:37.067 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:14:37.067 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:14:37.067 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (118956 ms total) 2024-09-16T17:14:37.068 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:14:37.137 INFO:tasks.workunit.client.0.smithi123.stdout:seed 46023 2024-09-16T17:14:37.137 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/3.* 2024-09-16T17:14:37.137 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:14:37.137 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:14:37.137 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/3, where TypeParam = LUKS1UnderLUKS1 2024-09-16T17:14:37.232 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/3.Simple 2024-09-16T17:14:38.942 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:15:07.924 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/3.Simple (30691 ms) 2024-09-16T17:15:07.925 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/3.Grow 2024-09-16T17:15:07.925 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:15:29.176 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/3.Grow (21252 ms) 2024-09-16T17:15:29.176 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/3.Shrink 2024-09-16T17:15:29.176 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:15:49.532 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/3.Shrink (20357 ms) 2024-09-16T17:15:49.532 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/3.ShrinkToOne 2024-09-16T17:15:49.532 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:16:07.791 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/3.ShrinkToOne (18259 ms) 2024-09-16T17:16:07.792 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/3.ShrinkToOneAfterSnapshot 2024-09-16T17:16:07.792 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:16:30.155 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/3.ShrinkToOneAfterSnapshot (22363 ms) 2024-09-16T17:16:30.155 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/3.MinOverlap 2024-09-16T17:16:30.155 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:16:47.969 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/3.MinOverlap (17815 ms) 2024-09-16T17:16:47.970 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/3.ZeroOverlap 2024-09-16T17:16:47.970 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:17:07.238 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/3.ZeroOverlap (19269 ms) 2024-09-16T17:17:08.091 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/3 (150006 ms total) 2024-09-16T17:17:08.091 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:17:08.091 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:17:08.091 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (150954 ms total) 2024-09-16T17:17:08.091 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:17:08.161 INFO:tasks.workunit.client.0.smithi123.stdout:seed 46094 2024-09-16T17:17:08.161 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/4.* 2024-09-16T17:17:08.161 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:17:08.161 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:17:08.161 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/4, where TypeParam = LUKS1UnderLUKS2 2024-09-16T17:17:08.230 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/4.Simple 2024-09-16T17:17:10.099 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:17:36.996 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/4.Simple (28766 ms) 2024-09-16T17:17:36.997 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/4.Grow 2024-09-16T17:17:36.997 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:18:03.950 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/4.Grow (26953 ms) 2024-09-16T17:18:03.950 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/4.Shrink 2024-09-16T17:18:03.950 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:18:31.590 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/4.Shrink (27641 ms) 2024-09-16T17:18:31.591 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/4.ShrinkToOne 2024-09-16T17:18:31.591 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:18:54.263 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/4.ShrinkToOne (22673 ms) 2024-09-16T17:18:54.263 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/4.ShrinkToOneAfterSnapshot 2024-09-16T17:18:54.264 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:19:22.911 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/4.ShrinkToOneAfterSnapshot (28647 ms) 2024-09-16T17:19:22.911 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/4.MinOverlap 2024-09-16T17:19:22.911 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:19:43.108 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/4.MinOverlap (20197 ms) 2024-09-16T17:19:43.108 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/4.ZeroOverlap 2024-09-16T17:19:43.108 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:20:02.691 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/4.ZeroOverlap (19584 ms) 2024-09-16T17:20:03.131 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/4 (174461 ms total) 2024-09-16T17:20:03.132 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:20:03.132 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:20:03.132 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (174970 ms total) 2024-09-16T17:20:03.132 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:20:03.224 INFO:tasks.workunit.client.0.smithi123.stdout:seed 52055 2024-09-16T17:20:03.226 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/5.* 2024-09-16T17:20:03.226 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:20:03.226 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:20:03.226 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/5, where TypeParam = UnformattedLUKS2 2024-09-16T17:20:03.316 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/5.Simple 2024-09-16T17:20:05.138 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:20:31.648 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/5.Simple (28331 ms) 2024-09-16T17:20:31.649 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/5.Grow 2024-09-16T17:20:31.649 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:20:53.185 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/5.Grow (21538 ms) 2024-09-16T17:20:53.186 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/5.Shrink 2024-09-16T17:20:53.186 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:21:18.674 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/5.Shrink (25488 ms) 2024-09-16T17:21:18.674 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/5.ShrinkToOne 2024-09-16T17:21:18.674 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:21:34.903 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/5.ShrinkToOne (16229 ms) 2024-09-16T17:21:34.903 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/5.ShrinkToOneAfterSnapshot 2024-09-16T17:21:34.903 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:21:58.132 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/5.ShrinkToOneAfterSnapshot (23229 ms) 2024-09-16T17:21:58.132 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/5.MinOverlap 2024-09-16T17:21:58.132 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:22:16.787 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/5.MinOverlap (18656 ms) 2024-09-16T17:22:16.788 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/5.ZeroOverlap 2024-09-16T17:22:16.788 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:22:35.168 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/5.ZeroOverlap (18381 ms) 2024-09-16T17:22:36.161 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/5 (151852 ms total) 2024-09-16T17:22:36.161 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:22:36.161 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:22:36.161 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (152935 ms total) 2024-09-16T17:22:36.161 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:22:36.249 INFO:tasks.workunit.client.0.smithi123.stdout:seed 58077 2024-09-16T17:22:36.250 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/6.* 2024-09-16T17:22:36.250 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:22:36.250 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:22:36.250 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/6, where TypeParam = LUKS2UnderLUKS2 2024-09-16T17:22:36.342 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/6.Simple 2024-09-16T17:22:38.166 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:23:14.938 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/6.Simple (38595 ms) 2024-09-16T17:23:14.938 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/6.Grow 2024-09-16T17:23:14.938 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:23:43.848 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/6.Grow (28911 ms) 2024-09-16T17:23:43.848 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/6.Shrink 2024-09-16T17:23:43.849 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:24:18.064 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/6.Shrink (34215 ms) 2024-09-16T17:24:18.064 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/6.ShrinkToOne 2024-09-16T17:24:18.064 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:24:38.259 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/6.ShrinkToOne (20196 ms) 2024-09-16T17:24:38.259 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/6.ShrinkToOneAfterSnapshot 2024-09-16T17:24:38.260 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:25:03.182 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/6.ShrinkToOneAfterSnapshot (24922 ms) 2024-09-16T17:25:03.182 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/6.MinOverlap 2024-09-16T17:25:03.182 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:25:30.613 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/6.MinOverlap (27431 ms) 2024-09-16T17:25:30.613 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/6.ZeroOverlap 2024-09-16T17:25:30.613 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:25:53.174 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/6.ZeroOverlap (22562 ms) 2024-09-16T17:25:53.203 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/6 (196832 ms total) 2024-09-16T17:25:53.203 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:25:53.204 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:25:53.204 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (196953 ms total) 2024-09-16T17:25:53.204 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:25:53.269 INFO:tasks.workunit.client.0.smithi123.stdout:seed 68868 2024-09-16T17:25:53.269 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = EncryptedFlattenTest/7.* 2024-09-16T17:25:53.269 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 7 tests from 1 test suite. 2024-09-16T17:25:53.269 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:25:53.269 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/7, where TypeParam = LUKS2UnderLUKS1 2024-09-16T17:25:53.376 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/7.Simple 2024-09-16T17:25:55.215 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:26:38.147 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/7.Simple (44770 ms) 2024-09-16T17:26:38.147 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/7.Grow 2024-09-16T17:26:38.147 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:27:05.249 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/7.Grow (27103 ms) 2024-09-16T17:27:05.249 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/7.Shrink 2024-09-16T17:27:05.249 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:27:36.653 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/7.Shrink (31404 ms) 2024-09-16T17:27:36.653 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/7.ShrinkToOne 2024-09-16T17:27:36.653 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:28:03.206 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/7.ShrinkToOne (26553 ms) 2024-09-16T17:28:03.207 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/7.ShrinkToOneAfterSnapshot 2024-09-16T17:28:03.207 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:28:37.255 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/7.ShrinkToOneAfterSnapshot (34048 ms) 2024-09-16T17:28:37.255 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/7.MinOverlap 2024-09-16T17:28:37.255 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:29:02.199 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/7.MinOverlap (24944 ms) 2024-09-16T17:29:02.199 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] EncryptedFlattenTest/7.ZeroOverlap 2024-09-16T17:29:02.200 INFO:tasks.workunit.client.0.smithi123.stdout:using new format! 2024-09-16T17:29:26.457 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] EncryptedFlattenTest/7.ZeroOverlap (24258 ms) 2024-09-16T17:29:27.245 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 7 tests from EncryptedFlattenTest/7 (213080 ms total) 2024-09-16T17:29:27.245 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:29:27.245 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:29:27.245 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 7 tests from 1 test suite ran. (213975 ms total) 2024-09-16T17:29:27.245 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 7 tests. 2024-09-16T17:29:27.329 INFO:tasks.workunit.client.0.smithi123.stdout:seed 74377 2024-09-16T17:29:27.330 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestImageWatcher.* 2024-09-16T17:29:27.330 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 17 tests from 1 test suite. 2024-09-16T17:29:27.330 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:29:27.330 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 17 tests from TestImageWatcher 2024-09-16T17:29:29.093 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyHeaderUpdate 2024-09-16T17:29:39.714 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyHeaderUpdate (10621 ms) 2024-09-16T17:29:39.714 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyFlatten 2024-09-16T17:29:40.424 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyFlatten (710 ms) 2024-09-16T17:29:40.424 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyResize 2024-09-16T17:29:41.159 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyResize (734 ms) 2024-09-16T17:29:41.159 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyRebuildObjectMap 2024-09-16T17:29:41.922 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyRebuildObjectMap (764 ms) 2024-09-16T17:29:41.923 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyUpdateFeatures 2024-09-16T17:29:42.324 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyUpdateFeatures (402 ms) 2024-09-16T17:29:42.324 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifySnapCreate 2024-09-16T17:29:42.753 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifySnapCreate (429 ms) 2024-09-16T17:29:42.754 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifySnapCreateError 2024-09-16T17:29:43.290 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifySnapCreateError (537 ms) 2024-09-16T17:29:43.290 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifySnapRename 2024-09-16T17:29:44.115 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifySnapRename (825 ms) 2024-09-16T17:29:44.115 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifySnapRenameError 2024-09-16T17:29:44.603 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifySnapRenameError (488 ms) 2024-09-16T17:29:44.603 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifySnapRemove 2024-09-16T17:29:45.019 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifySnapRemove (416 ms) 2024-09-16T17:29:45.019 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifySnapProtect 2024-09-16T17:29:45.489 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifySnapProtect (470 ms) 2024-09-16T17:29:45.489 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifySnapUnprotect 2024-09-16T17:29:45.943 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifySnapUnprotect (454 ms) 2024-09-16T17:29:45.943 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyRename 2024-09-16T17:29:46.470 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyRename (527 ms) 2024-09-16T17:29:46.470 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyAsyncTimedOut 2024-09-16T17:29:46.819 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyAsyncTimedOut (349 ms) 2024-09-16T17:29:46.819 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyAsyncError 2024-09-16T17:29:47.327 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyAsyncError (508 ms) 2024-09-16T17:29:47.327 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyAsyncCompleteError 2024-09-16T17:29:47.753 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyAsyncCompleteError (426 ms) 2024-09-16T17:29:47.753 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestImageWatcher.NotifyAsyncRequestTimedOut 2024-09-16T17:29:48.196 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestImageWatcher.NotifyAsyncRequestTimedOut (443 ms) 2024-09-16T17:29:49.099 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 17 tests from TestImageWatcher (19103 ms total) 2024-09-16T17:29:49.099 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:29:49.099 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:29:49.100 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 17 tests from 1 test suite ran. (21770 ms total) 2024-09-16T17:29:49.100 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 17 tests. 2024-09-16T17:29:49.158 INFO:tasks.workunit.client.0.smithi123.stdout:seed 74445 2024-09-16T17:29:49.159 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestInternal.* 2024-09-16T17:29:49.159 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 34 tests from 1 test suite. 2024-09-16T17:29:49.159 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:29:49.159 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 34 tests from TestInternal 2024-09-16T17:29:51.104 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.OpenByID 2024-09-16T17:29:53.483 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.OpenByID (2379 ms) 2024-09-16T17:29:53.484 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.OpenSnapDNE 2024-09-16T17:29:53.809 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.OpenSnapDNE (326 ms) 2024-09-16T17:29:53.809 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.IsExclusiveLockOwner 2024-09-16T17:29:54.130 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.IsExclusiveLockOwner (321 ms) 2024-09-16T17:29:54.131 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.ResizeLocksImage 2024-09-16T17:29:54.487 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.ResizeLocksImage (357 ms) 2024-09-16T17:29:54.488 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.ResizeFailsToLockImage 2024-09-16T17:29:54.731 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.ResizeFailsToLockImage (244 ms) 2024-09-16T17:29:54.731 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapCreateLocksImage 2024-09-16T17:29:56.277 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SnapCreateLocksImage (1546 ms) 2024-09-16T17:29:56.277 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapCreateFailsToLockImage 2024-09-16T17:29:56.537 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SnapCreateFailsToLockImage (260 ms) 2024-09-16T17:29:56.537 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapRollbackLocksImage 2024-09-16T17:29:58.323 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SnapRollbackLocksImage (1786 ms) 2024-09-16T17:29:58.323 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapRollbackFailsToLockImage 2024-09-16T17:30:00.299 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SnapRollbackFailsToLockImage (1976 ms) 2024-09-16T17:30:00.299 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapSetReleasesLock 2024-09-16T17:30:02.299 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SnapSetReleasesLock (2000 ms) 2024-09-16T17:30:02.299 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.FlattenLocksImage 2024-09-16T17:30:05.364 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.FlattenLocksImage (3065 ms) 2024-09-16T17:30:05.364 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.FlattenFailsToLockImage 2024-09-16T17:30:09.322 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.FlattenFailsToLockImage (3958 ms) 2024-09-16T17:30:09.322 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.WriteFailsToLockImageBlocklisted 2024-09-16T17:30:09.423 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_internal.cc:382: Skipped 2024-09-16T17:30:09.424 INFO:tasks.workunit.client.0.smithi123.stdout:Not supported by crimson yet. Skipped 2024-09-16T17:30:09.424 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.WriteFailsToLockImageBlocklisted (101 ms) 2024-09-16T17:30:09.424 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.WriteFailsToLockImageBlocklistedWatch 2024-09-16T17:30:09.523 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_internal.cc:416: Skipped 2024-09-16T17:30:09.523 INFO:tasks.workunit.client.0.smithi123.stdout:Not supported by crimson yet. Skipped 2024-09-16T17:30:09.523 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.WriteFailsToLockImageBlocklistedWatch (100 ms) 2024-09-16T17:30:09.523 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.AioWriteRequestsLock 2024-09-16T17:30:09.952 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.AioWriteRequestsLock (429 ms) 2024-09-16T17:30:09.953 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.AioDiscardRequestsLock 2024-09-16T17:30:10.327 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.AioDiscardRequestsLock (375 ms) 2024-09-16T17:30:10.327 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.CancelAsyncResize 2024-09-16T17:30:11.017 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.CancelAsyncResize (690 ms) 2024-09-16T17:30:11.017 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.MultipleResize 2024-09-16T17:30:12.023 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.MultipleResize (1006 ms) 2024-09-16T17:30:12.023 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.Metadata 2024-09-16T17:30:12.888 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.Metadata (865 ms) 2024-09-16T17:30:12.888 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.MetadataConfApply 2024-09-16T17:30:13.402 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.MetadataConfApply (514 ms) 2024-09-16T17:30:13.403 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapshotCopyup 2024-09-16T17:30:13.503 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_internal.cc:662: Skipped 2024-09-16T17:30:13.503 INFO:tasks.workunit.client.0.smithi123.stdout:Not supported by crimson yet. Skipped 2024-09-16T17:30:13.503 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.SnapshotCopyup (100 ms) 2024-09-16T17:30:13.503 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapshotCopyupZeros 2024-09-16T17:30:16.499 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SnapshotCopyupZeros (2997 ms) 2024-09-16T17:30:16.499 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SnapshotCopyupZerosMigration 2024-09-16T17:30:18.474 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SnapshotCopyupZerosMigration (1975 ms) 2024-09-16T17:30:18.475 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.ResizeCopyup 2024-09-16T17:30:21.044 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.ResizeCopyup (2570 ms) 2024-09-16T17:30:21.044 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.DiscardCopyup 2024-09-16T17:30:21.153 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_internal.cc:1055: Skipped 2024-09-16T17:30:21.153 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:30:21.153 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.DiscardCopyup (109 ms) 2024-09-16T17:30:21.153 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.ImageOptions 2024-09-16T17:30:21.274 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.ImageOptions (121 ms) 2024-09-16T17:30:21.274 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.WriteFullCopyup 2024-09-16T17:30:28.333 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.WriteFullCopyup (7059 ms) 2024-09-16T17:30:28.333 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.DiffIterateCloneOverwrite 2024-09-16T17:30:30.481 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.DiffIterateCloneOverwrite (2147 ms) 2024-09-16T17:30:30.481 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.TestCoR 2024-09-16T17:30:30.604 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_internal.cc:1312: Skipped 2024-09-16T17:30:30.604 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to disabled copy-on-read 2024-09-16T17:30:30.604 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.TestCoR (123 ms) 2024-09-16T17:30:30.604 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.FlattenNoEmptyObjects 2024-09-16T17:30:35.795 INFO:tasks.workunit.client.0.smithi123.stdout:made parent image "image39": 4194304K (1024 * 4K) 2024-09-16T17:30:35.795 INFO:tasks.workunit.client.0.smithi123.stdout:generated random write map: 2024-09-16T17:30:35.795 INFO:tasks.workunit.client.0.smithi123.stdout: [6 , 25374 ] 2024-09-16T17:30:35.795 INFO:tasks.workunit.client.0.smithi123.stdout: [7 , 28813 ] 2024-09-16T17:30:35.795 INFO:tasks.workunit.client.0.smithi123.stdout: [9 , 37611 ] 2024-09-16T17:30:35.795 INFO:tasks.workunit.client.0.smithi123.stdout: [14 , 60292 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [19 , 79388 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [25 , 104960 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [36 , 150787 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [66 , 273832 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [71 , 292251 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [77 , 315394 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [98 , 402750 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [106 , 437363 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [122 , 501117 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [129 , 530063 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [159 , 652243 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [173 , 710473 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [178 , 729262 ] 2024-09-16T17:30:35.796 INFO:tasks.workunit.client.0.smithi123.stdout: [180 , 739542 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [204 , 838606 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [212 , 868997 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [222 , 911893 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [223 , 915209 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [228 , 935806 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [237 , 973332 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [246 , 1010375 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [271 , 1112311 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [290 , 1188520 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [292 , 1199185 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [304 , 1248541 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [339 , 1391689 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [351 , 1437772 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [354 , 1452537 ] 2024-09-16T17:30:35.797 INFO:tasks.workunit.client.0.smithi123.stdout: [369 , 1512564 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [375 , 1538960 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [385 , 1578073 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [389 , 1596188 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [396 , 1624442 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [408 , 1673519 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [469 , 1922190 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [497 , 2036933 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [499 , 2047162 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [500 , 2051500 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [513 , 2101407 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [523 , 2142690 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [524 , 2147802 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [543 , 2225314 ] 2024-09-16T17:30:35.798 INFO:tasks.workunit.client.0.smithi123.stdout: [546 , 2237037 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [550 , 2254934 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [551 , 2257283 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [563 , 2306269 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [567 , 2323360 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [569 , 2332001 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [580 , 2376580 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [582 , 2387411 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [606 , 2484477 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [609 , 2495747 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [620 , 2541516 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [623 , 2552483 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [639 , 2618487 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [640 , 2623706 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [646 , 2646704 ] 2024-09-16T17:30:35.799 INFO:tasks.workunit.client.0.smithi123.stdout: [655 , 2684771 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [669 , 2742570 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [675 , 2766934 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [682 , 2794732 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [683 , 2800374 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [696 , 2851998 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [703 , 2880426 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [714 , 2924659 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [726 , 2973973 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [731 , 2995548 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [738 , 3023878 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [758 , 3105729 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [765 , 3134113 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [774 , 3172669 ] 2024-09-16T17:30:35.800 INFO:tasks.workunit.client.0.smithi123.stdout: [778 , 3187116 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [806 , 3302417 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [815 , 3341342 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [828 , 3393210 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [835 , 3423065 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [845 , 3462595 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [868 , 3555631 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [882 , 3613335 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [888 , 3639336 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [891 , 3652540 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [905 , 3709840 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [912 , 3738632 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [926 , 3796304 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [939 , 3847318 ] 2024-09-16T17:30:35.801 INFO:tasks.workunit.client.0.smithi123.stdout: [947 , 3881027 ] 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: [950 , 3892216 ] 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: [952 , 3901065 ] 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: [955 , 3913979 ] 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: [957 , 3921243 ] 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: [1001 , 4102252 ] 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: [1011 , 4144504 ] 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout:write data based on random map 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-6 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-7 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-9 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-14 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-19 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-25 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-36 2024-09-16T17:30:35.802 INFO:tasks.workunit.client.0.smithi123.stdout: write object-66 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-71 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-77 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-98 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-106 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-122 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-129 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-159 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-173 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-178 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-180 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-204 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-212 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-222 2024-09-16T17:30:35.803 INFO:tasks.workunit.client.0.smithi123.stdout: write object-223 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-228 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-237 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-246 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-271 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-290 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-292 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-304 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-339 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-351 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-354 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-369 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-375 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-385 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-389 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-396 2024-09-16T17:30:35.804 INFO:tasks.workunit.client.0.smithi123.stdout: write object-408 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-469 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-497 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-499 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-500 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-513 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-523 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-524 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-543 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-546 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-550 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-551 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-563 2024-09-16T17:30:35.805 INFO:tasks.workunit.client.0.smithi123.stdout: write object-567 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-569 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-580 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-582 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-606 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-609 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-620 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-623 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-639 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-640 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-646 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-655 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-669 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-675 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-682 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-683 2024-09-16T17:30:35.806 INFO:tasks.workunit.client.0.smithi123.stdout: write object-696 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-703 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-714 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-726 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-731 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-738 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-758 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-765 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-774 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-778 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-806 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-815 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-828 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-835 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-845 2024-09-16T17:30:35.807 INFO:tasks.workunit.client.0.smithi123.stdout: write object-868 2024-09-16T17:30:35.808 INFO:tasks.workunit.client.0.smithi123.stdout: write object-882 2024-09-16T17:30:35.808 INFO:tasks.workunit.client.0.smithi123.stdout: write object-888 2024-09-16T17:30:35.808 INFO:tasks.workunit.client.0.smithi123.stdout: write object-891 2024-09-16T17:30:35.808 INFO:tasks.workunit.client.0.smithi123.stdout: write object-905 2024-09-16T17:30:35.808 INFO:tasks.workunit.client.0.smithi123.stdout: write object-912 2024-09-16T17:30:35.808 INFO:tasks.workunit.client.0.smithi123.stdout: write object-926 2024-09-16T17:30:42.457 INFO:tasks.workunit.client.0.smithi123.stdout: write object-939 2024-09-16T17:30:42.457 INFO:tasks.workunit.client.0.smithi123.stdout: write object-947 2024-09-16T17:30:42.457 INFO:tasks.workunit.client.0.smithi123.stdout: write object-950 2024-09-16T17:30:42.457 INFO:tasks.workunit.client.0.smithi123.stdout: write object-952 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout: write object-955 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout: write object-957 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout: write object-1001 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout: write object-1011 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout:verify written data by reading 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout: read object-6 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout:made snapshot "image39@parent_snap" and protect it 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout:made and opened clone "image40" 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout:flattening clone: "image40" 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout:check whether child image has the same set of objects as parent 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.FlattenNoEmptyObjects (11854 ms) 2024-09-16T17:30:42.458 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.PoolMetadataConfApply 2024-09-16T17:30:43.464 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.PoolMetadataConfApply (1007 ms) 2024-09-16T17:30:43.465 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.Sparsify 2024-09-16T17:30:43.871 INFO:tasks.workunit.client.0.smithi123.stdout:sparsify_supported=1 2024-09-16T17:30:43.872 INFO:tasks.workunit.client.0.smithi123.stdout:sparse_read_supported=1 2024-09-16T17:30:44.563 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.Sparsify (1099 ms) 2024-09-16T17:30:44.563 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.SparsifyClone 2024-09-16T17:30:44.834 INFO:tasks.workunit.client.0.smithi123.stdout:sparsify_supported=1 2024-09-16T17:30:48.269 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.SparsifyClone (3705 ms) 2024-09-16T17:30:48.269 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestInternal.MissingDataPool 2024-09-16T17:30:50.827 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestInternal.MissingDataPool (2559 ms) 2024-09-16T17:30:51.269 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 34 tests from TestInternal (59723 ms total) 2024-09-16T17:30:51.269 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:30:51.269 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:30:51.269 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 34 tests from 1 test suite ran. (62109 ms total) 2024-09-16T17:30:51.269 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 29 tests. 2024-09-16T17:30:51.270 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] 5 tests, listed below: 2024-09-16T17:30:51.270 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.WriteFailsToLockImageBlocklisted 2024-09-16T17:30:51.270 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.WriteFailsToLockImageBlocklistedWatch 2024-09-16T17:30:51.270 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.SnapshotCopyup 2024-09-16T17:30:51.270 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.DiscardCopyup 2024-09-16T17:30:51.270 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestInternal.TestCoR 2024-09-16T17:30:51.342 INFO:tasks.workunit.client.0.smithi123.stdout:seed 74502 2024-09-16T17:30:51.343 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestMirroring.* 2024-09-16T17:30:51.343 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 51 tests from 1 test suite. 2024-09-16T17:30:51.343 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:30:51.343 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 51 tests from TestMirroring 2024-09-16T17:30:53.107 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeImage 2024-09-16T17:30:56.959 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeImage (3852 ms) 2024-09-16T17:30:56.960 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModePool 2024-09-16T17:30:58.729 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModePool (1770 ms) 2024-09-16T17:30:58.729 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled 2024-09-16T17:31:00.062 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled (1333 ms) 2024-09-16T17:31:00.062 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModeImage 2024-09-16T17:31:01.878 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModeImage (1816 ms) 2024-09-16T17:31:01.879 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModeImage_NoObjectMap 2024-09-16T17:31:03.489 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModeImage_NoObjectMap (1611 ms) 2024-09-16T17:31:03.490 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModePool 2024-09-16T17:31:05.083 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModePool (1594 ms) 2024-09-16T17:31:05.084 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableImageMirror_In_MirrorModeDisabled 2024-09-16T17:31:07.041 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.DisableImageMirror_In_MirrorModeDisabled (1957 ms) 2024-09-16T17:31:07.042 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableImageMirrorWithPeer 2024-09-16T17:31:07.042 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_mirroring.cc:482: Skipped 2024-09-16T17:31:07.042 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:31:07.042 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestMirroring.DisableImageMirrorWithPeer (0 ms) 2024-09-16T17:31:07.042 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableJournalingWithPeer 2024-09-16T17:31:07.042 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_mirroring.cc:520: Skipped 2024-09-16T17:31:07.043 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:31:07.043 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestMirroring.DisableJournalingWithPeer (1 ms) 2024-09-16T17:31:07.043 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled_WithoutJournaling 2024-09-16T17:31:08.191 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeDisabled_WithoutJournaling (1150 ms) 2024-09-16T17:31:08.191 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModePool_WithoutJournaling 2024-09-16T17:31:09.544 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModePool_WithoutJournaling (1353 ms) 2024-09-16T17:31:09.544 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutJournaling 2024-09-16T17:31:11.920 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutJournaling (2375 ms) 2024-09-16T17:31:11.920 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutExclusiveLock 2024-09-16T17:31:13.614 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableImageMirror_In_MirrorModeImage_WithoutExclusiveLock (1695 ms) 2024-09-16T17:31:13.614 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModeDisabled 2024-09-16T17:31:15.062 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModeDisabled (1448 ms) 2024-09-16T17:31:15.063 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModeImage 2024-09-16T17:31:16.577 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModeImage (1514 ms) 2024-09-16T17:31:16.577 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModePool 2024-09-16T17:31:18.282 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModePool (1706 ms) 2024-09-16T17:31:18.282 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModePool_WithoutJournaling 2024-09-16T17:31:19.730 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModePool_WithoutJournaling (1448 ms) 2024-09-16T17:31:19.731 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.CreateImage_In_MirrorModeImage_WithoutJournaling 2024-09-16T17:31:20.987 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.CreateImage_In_MirrorModeImage_WithoutJournaling (1257 ms) 2024-09-16T17:31:20.988 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModeDisabled 2024-09-16T17:31:22.584 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModeDisabled (1597 ms) 2024-09-16T17:31:22.584 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModeImage 2024-09-16T17:31:24.302 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModeImage (1718 ms) 2024-09-16T17:31:24.302 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModeImage_SnapshotMirroringEnabled 2024-09-16T17:31:28.000 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModeImage_SnapshotMirroringEnabled (3697 ms) 2024-09-16T17:31:28.000 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.EnableJournaling_In_MirrorModePool 2024-09-16T17:31:29.797 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.EnableJournaling_In_MirrorModePool (1798 ms) 2024-09-16T17:31:29.797 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableJournaling_In_MirrorModePool 2024-09-16T17:31:31.511 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.DisableJournaling_In_MirrorModePool (1713 ms) 2024-09-16T17:31:31.511 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.DisableJournaling_In_MirrorModeImage 2024-09-16T17:31:33.172 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.DisableJournaling_In_MirrorModeImage (1662 ms) 2024-09-16T17:31:33.172 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.MirrorModeSet_DisabledMode_To_PoolMode 2024-09-16T17:31:35.709 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.MirrorModeSet_DisabledMode_To_PoolMode (2537 ms) 2024-09-16T17:31:35.709 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.MirrorModeSet_PoolMode_To_DisabledMode 2024-09-16T17:31:38.475 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.MirrorModeSet_PoolMode_To_DisabledMode (2765 ms) 2024-09-16T17:31:38.475 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.MirrorModeSet_DisabledMode_To_ImageMode 2024-09-16T17:31:40.652 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.MirrorModeSet_DisabledMode_To_ImageMode (2178 ms) 2024-09-16T17:31:40.652 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.MirrorModeSet_PoolMode_To_ImageMode 2024-09-16T17:31:43.183 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.MirrorModeSet_PoolMode_To_ImageMode (2531 ms) 2024-09-16T17:31:43.183 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.MirrorModeSet_ImageMode_To_PoolMode 2024-09-16T17:31:45.932 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.MirrorModeSet_ImageMode_To_PoolMode (2749 ms) 2024-09-16T17:31:45.932 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.MirrorModeSet_ImageMode_To_DisabledMode 2024-09-16T17:31:48.632 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.MirrorModeSet_ImageMode_To_DisabledMode (2700 ms) 2024-09-16T17:31:48.633 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.RemoveImage_With_MirrorImageEnabled 2024-09-16T17:31:50.206 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.RemoveImage_With_MirrorImageEnabled (1574 ms) 2024-09-16T17:31:50.207 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.RemoveImage_With_MirrorImageDisabled 2024-09-16T17:31:51.489 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.RemoveImage_With_MirrorImageDisabled (1283 ms) 2024-09-16T17:31:51.489 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.RemoveImage_With_ImageWithoutJournal 2024-09-16T17:31:52.548 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.RemoveImage_With_ImageWithoutJournal (1059 ms) 2024-09-16T17:31:52.549 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.RemoveImage_With_MirrorImageDemoted 2024-09-16T17:31:54.690 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.RemoveImage_With_MirrorImageDemoted (2142 ms) 2024-09-16T17:31:54.691 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.TrashMoveRestore_PoolMode 2024-09-16T17:31:56.710 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.TrashMoveRestore_PoolMode (2019 ms) 2024-09-16T17:31:56.710 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.TrashMoveRestore_ImageMode_MirroringDisabled 2024-09-16T17:31:58.775 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.TrashMoveRestore_ImageMode_MirroringDisabled (2065 ms) 2024-09-16T17:31:58.775 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.TrashMoveRestore_ImageMode_MirroringEnabled 2024-09-16T17:32:00.720 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.TrashMoveRestore_ImageMode_MirroringEnabled (1946 ms) 2024-09-16T17:32:00.720 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.MirrorStatusList 2024-09-16T17:32:02.315 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.MirrorStatusList (1595 ms) 2024-09-16T17:32:02.316 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.RemoveBootstrapped 2024-09-16T17:32:03.955 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.RemoveBootstrapped (1640 ms) 2024-09-16T17:32:03.956 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.AioPromoteDemote 2024-09-16T17:32:12.760 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.AioPromoteDemote (8804 ms) 2024-09-16T17:32:12.760 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.AioGetInfo 2024-09-16T17:32:19.220 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.AioGetInfo (6461 ms) 2024-09-16T17:32:19.220 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.AioGetStatus 2024-09-16T17:32:23.774 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.AioGetStatus (4554 ms) 2024-09-16T17:32:23.774 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.SiteName 2024-09-16T17:32:23.783 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.SiteName (9 ms) 2024-09-16T17:32:23.784 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.Bootstrap 2024-09-16T17:32:39.507 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.Bootstrap (15723 ms) 2024-09-16T17:32:39.507 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.PeerDirection 2024-09-16T17:32:39.601 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.PeerDirection (94 ms) 2024-09-16T17:32:39.601 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.Snapshot 2024-09-16T17:32:56.387 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.Snapshot (16786 ms) 2024-09-16T17:32:56.387 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.SnapshotRemoveOnDisable 2024-09-16T17:33:01.472 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.SnapshotRemoveOnDisable (5085 ms) 2024-09-16T17:33:01.473 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.SnapshotUnlinkPeer 2024-09-16T17:33:15.056 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.SnapshotUnlinkPeer (13584 ms) 2024-09-16T17:33:15.057 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.SnapshotImageState 2024-09-16T17:33:23.868 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.SnapshotImageState (8811 ms) 2024-09-16T17:33:23.868 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.SnapshotPromoteDemote 2024-09-16T17:33:35.898 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.SnapshotPromoteDemote (12031 ms) 2024-09-16T17:33:35.899 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroring.AioSnapshotCreate 2024-09-16T17:34:18.940 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroring.AioSnapshotCreate (43042 ms) 2024-09-16T17:34:19.876 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 51 tests from TestMirroring (205833 ms total) 2024-09-16T17:34:19.876 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:34:19.876 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:34:19.877 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 51 tests from 1 test suite ran. (208533 ms total) 2024-09-16T17:34:19.877 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 49 tests. 2024-09-16T17:34:19.877 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] 2 tests, listed below: 2024-09-16T17:34:19.877 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestMirroring.DisableImageMirrorWithPeer 2024-09-16T17:34:19.877 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestMirroring.DisableJournalingWithPeer 2024-09-16T17:34:20.015 INFO:tasks.workunit.client.0.smithi123.stdout:seed 75007 2024-09-16T17:34:20.015 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestGroup.* 2024-09-16T17:34:20.015 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 13 tests from 1 test suite. 2024-09-16T17:34:20.016 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:34:20.016 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 13 tests from TestGroup 2024-09-16T17:34:21.874 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.group_create 2024-09-16T17:34:24.278 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.group_create (2404 ms) 2024-09-16T17:34:24.278 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.group_createPP 2024-09-16T17:34:24.608 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.group_createPP (327 ms) 2024-09-16T17:34:24.609 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.group_get_id 2024-09-16T17:34:24.851 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.group_get_id (246 ms) 2024-09-16T17:34:24.851 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.group_get_idPP 2024-09-16T17:34:25.106 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.group_get_idPP (255 ms) 2024-09-16T17:34:25.106 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.add_image 2024-09-16T17:34:25.862 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.add_image (756 ms) 2024-09-16T17:34:25.862 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.add_imagePP 2024-09-16T17:34:26.500 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.add_imagePP (637 ms) 2024-09-16T17:34:26.500 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.add_snapshot 2024-09-16T17:34:33.627 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.add_snapshot (7128 ms) 2024-09-16T17:34:33.628 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.add_snapshotPP 2024-09-16T17:34:41.637 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.add_snapshotPP (8009 ms) 2024-09-16T17:34:41.637 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.snap_get_info 2024-09-16T17:34:47.393 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.snap_get_info (5757 ms) 2024-09-16T17:34:47.394 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.snap_get_infoPP 2024-09-16T17:34:53.167 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.snap_get_infoPP (5773 ms) 2024-09-16T17:34:53.167 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.snap_list2 2024-09-16T17:35:04.218 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.snap_list2 (11052 ms) 2024-09-16T17:35:04.219 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.snap_list2PP 2024-09-16T17:35:15.267 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.snap_list2PP (11048 ms) 2024-09-16T17:35:15.267 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestGroup.snap_list_internal 2024-09-16T17:35:17.817 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestGroup.snap_list_internal (2551 ms) 2024-09-16T17:35:18.159 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 13 tests from TestGroup (55943 ms total) 2024-09-16T17:35:18.159 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:18.159 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:35:18.159 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 13 tests from 1 test suite ran. (58144 ms total) 2024-09-16T17:35:18.159 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 13 tests. 2024-09-16T17:35:18.228 INFO:tasks.workunit.client.0.smithi123.stdout:seed 75071 2024-09-16T17:35:18.229 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestMirroringWatcher.* 2024-09-16T17:35:18.229 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 2 tests from 1 test suite. 2024-09-16T17:35:18.229 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:35:18.229 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 2 tests from TestMirroringWatcher 2024-09-16T17:35:20.329 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroringWatcher.ModeUpdated 2024-09-16T17:35:22.770 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroringWatcher.ModeUpdated (2441 ms) 2024-09-16T17:35:22.770 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestMirroringWatcher.ImageStatusUpdated 2024-09-16T17:35:23.062 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestMirroringWatcher.ImageStatusUpdated (292 ms) 2024-09-16T17:35:23.345 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 2 tests from TestMirroringWatcher (2733 ms total) 2024-09-16T17:35:23.346 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:23.346 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:35:23.346 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 2 tests from 1 test suite ran. (5116 ms total) 2024-09-16T17:35:23.346 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 2 tests. 2024-09-16T17:35:23.425 INFO:tasks.workunit.client.0.smithi123.stdout:seed 75123 2024-09-16T17:35:23.426 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestObjectMap.* 2024-09-16T17:35:23.426 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 4 tests from 1 test suite. 2024-09-16T17:35:23.426 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:35:23.426 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 4 tests from TestObjectMap 2024-09-16T17:35:25.363 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestObjectMap.RefreshInvalidatesWhenCorrupt 2024-09-16T17:35:28.087 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestObjectMap.RefreshInvalidatesWhenCorrupt (2724 ms) 2024-09-16T17:35:28.088 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestObjectMap.RefreshInvalidatesWhenTooSmall 2024-09-16T17:35:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestObjectMap.RefreshInvalidatesWhenTooSmall (625 ms) 2024-09-16T17:35:28.713 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestObjectMap.InvalidateFlagOnDisk 2024-09-16T17:35:29.268 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestObjectMap.InvalidateFlagOnDisk (556 ms) 2024-09-16T17:35:29.268 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestObjectMap.AcquireLockInvalidatesWhenTooSmall 2024-09-16T17:35:29.744 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestObjectMap.AcquireLockInvalidatesWhenTooSmall (476 ms) 2024-09-16T17:35:30.381 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 4 tests from TestObjectMap (4381 ms total) 2024-09-16T17:35:30.381 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:30.382 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:35:30.382 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 4 tests from 1 test suite ran. (6955 ms total) 2024-09-16T17:35:30.382 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 4 tests. 2024-09-16T17:35:30.382 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:30.382 INFO:tasks.workunit.client.0.smithi123.stdout: YOU HAVE 1 DISABLED TEST 2024-09-16T17:35:30.382 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:30.450 INFO:tasks.workunit.client.0.smithi123.stdout:seed 75175 2024-09-16T17:35:30.451 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestOperations.* 2024-09-16T17:35:30.451 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 1 test from 1 test suite. 2024-09-16T17:35:30.451 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:35:30.451 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 1 test from TestOperations 2024-09-16T17:35:32.383 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestOperations.DisableJournalingCorrupt 2024-09-16T17:35:34.387 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/test_Operations.cc:18: Skipped 2024-09-16T17:35:34.387 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:34.387 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestOperations.DisableJournalingCorrupt (2004 ms) 2024-09-16T17:35:35.174 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 1 test from TestOperations (2004 ms total) 2024-09-16T17:35:35.174 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:35.174 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:35:35.174 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 1 test from 1 test suite ran. (4723 ms total) 2024-09-16T17:35:35.174 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 0 tests. 2024-09-16T17:35:35.174 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] 1 test, listed below: 2024-09-16T17:35:35.174 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestOperations.DisableJournalingCorrupt 2024-09-16T17:35:35.255 INFO:tasks.workunit.client.0.smithi123.stdout:seed 75225 2024-09-16T17:35:35.256 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestTrash.* 2024-09-16T17:35:35.256 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 2 tests from 1 test suite. 2024-09-16T17:35:35.256 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:35:35.256 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 2 tests from TestTrash 2024-09-16T17:35:37.181 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestTrash.UserRemovingSource 2024-09-16T17:35:40.819 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestTrash.UserRemovingSource (3637 ms) 2024-09-16T17:35:40.819 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestTrash.RestoreMirroringSource 2024-09-16T17:35:41.601 INFO:tasks.workunit.client.0.smithi123.stdout:[ OK ] TestTrash.RestoreMirroringSource (782 ms) 2024-09-16T17:35:42.314 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 2 tests from TestTrash (4420 ms total) 2024-09-16T17:35:42.314 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:42.314 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:35:42.314 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 2 tests from 1 test suite ran. (7058 ms total) 2024-09-16T17:35:42.314 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 2 tests. 2024-09-16T17:35:42.382 INFO:tasks.workunit.client.0.smithi123.stdout:seed 75278 2024-09-16T17:35:42.383 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestJournalEntries.* 2024-09-16T17:35:42.383 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 4 tests from 1 test suite. 2024-09-16T17:35:42.383 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:35:42.384 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 4 tests from TestJournalEntries 2024-09-16T17:35:44.173 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalEntries.AioWrite 2024-09-16T17:35:46.605 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Entries.cc:118: Skipped 2024-09-16T17:35:46.605 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:46.605 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioWrite (2432 ms) 2024-09-16T17:35:46.605 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalEntries.AioDiscard 2024-09-16T17:35:46.755 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Entries.cc:163: Skipped 2024-09-16T17:35:46.755 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:46.755 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioDiscard (150 ms) 2024-09-16T17:35:46.755 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalEntries.AioDiscardWithPrune 2024-09-16T17:35:46.909 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Entries.cc:200: Skipped 2024-09-16T17:35:46.909 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:46.909 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioDiscardWithPrune (155 ms) 2024-09-16T17:35:46.909 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalEntries.AioFlush 2024-09-16T17:35:47.076 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Entries.cc:263: Skipped 2024-09-16T17:35:47.076 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:47.076 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioFlush (167 ms) 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 4 tests from TestJournalEntries (2904 ms total) 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 4 tests from 1 test suite ran. (4990 ms total) 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 0 tests. 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] 4 tests, listed below: 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioWrite 2024-09-16T17:35:47.373 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioDiscard 2024-09-16T17:35:47.374 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioDiscardWithPrune 2024-09-16T17:35:47.374 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalEntries.AioFlush 2024-09-16T17:35:47.435 INFO:tasks.workunit.client.0.smithi123.stdout:seed 75327 2024-09-16T17:35:47.435 INFO:tasks.workunit.client.0.smithi123.stdout:Note: Google Test filter = TestJournalReplay.* 2024-09-16T17:35:47.436 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] Running 16 tests from 1 test suite. 2024-09-16T17:35:47.436 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment set-up. 2024-09-16T17:35:47.436 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 16 tests from TestJournalReplay 2024-09-16T17:35:49.383 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.AioDiscardEvent 2024-09-16T17:35:51.858 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:109: Skipped 2024-09-16T17:35:51.858 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:51.858 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.AioDiscardEvent (2475 ms) 2024-09-16T17:35:51.858 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.AioWriteEvent 2024-09-16T17:35:52.019 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:200: Skipped 2024-09-16T17:35:52.020 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:52.020 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.AioWriteEvent (162 ms) 2024-09-16T17:35:52.020 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.AioFlushEvent 2024-09-16T17:35:52.182 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:261: Skipped 2024-09-16T17:35:52.182 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:52.182 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.AioFlushEvent (162 ms) 2024-09-16T17:35:52.182 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.SnapCreate 2024-09-16T17:35:52.387 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:307: Skipped 2024-09-16T17:35:52.387 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:52.387 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapCreate (205 ms) 2024-09-16T17:35:52.387 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.SnapProtect 2024-09-16T17:35:52.544 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:348: Skipped 2024-09-16T17:35:52.544 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:52.544 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapProtect (157 ms) 2024-09-16T17:35:52.544 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.SnapUnprotect 2024-09-16T17:35:52.736 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:394: Skipped 2024-09-16T17:35:52.736 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:52.736 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapUnprotect (192 ms) 2024-09-16T17:35:52.736 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.SnapRename 2024-09-16T17:35:52.958 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:450: Skipped 2024-09-16T17:35:52.958 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:52.958 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapRename (222 ms) 2024-09-16T17:35:52.958 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.SnapRollback 2024-09-16T17:35:53.133 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:500: Skipped 2024-09-16T17:35:53.134 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:53.134 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapRollback (175 ms) 2024-09-16T17:35:53.134 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.SnapRemove 2024-09-16T17:35:53.300 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:541: Skipped 2024-09-16T17:35:53.300 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:53.300 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapRemove (167 ms) 2024-09-16T17:35:53.300 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.Rename 2024-09-16T17:35:53.492 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:590: Skipped 2024-09-16T17:35:53.492 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:53.493 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.Rename (192 ms) 2024-09-16T17:35:53.493 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.Resize 2024-09-16T17:35:53.684 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:624: Skipped 2024-09-16T17:35:53.684 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:53.684 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.Resize (191 ms) 2024-09-16T17:35:53.684 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.Flatten 2024-09-16T17:35:53.903 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:657: Skipped 2024-09-16T17:35:53.903 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:53.903 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.Flatten (220 ms) 2024-09-16T17:35:53.904 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.UpdateFeatures 2024-09-16T17:35:54.086 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:704: Skipped 2024-09-16T17:35:54.086 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:54.086 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.UpdateFeatures (183 ms) 2024-09-16T17:35:54.086 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.MetadataSet 2024-09-16T17:35:54.255 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:741: Skipped 2024-09-16T17:35:54.255 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:54.255 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.MetadataSet (169 ms) 2024-09-16T17:35:54.255 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.MetadataRemove 2024-09-16T17:35:54.419 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:781: Skipped 2024-09-16T17:35:54.420 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:54.420 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.MetadataRemove (165 ms) 2024-09-16T17:35:54.420 INFO:tasks.workunit.client.0.smithi123.stdout:[ RUN ] TestJournalReplay.ObjectPosition 2024-09-16T17:35:54.574 INFO:tasks.workunit.client.0.smithi123.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.3.0-4879-g81fedcb3/rpm/el9/BUILD/ceph-19.3.0-4879-g81fedcb3/src/test/librbd/journal/test_Replay.cc:824: Skipped 2024-09-16T17:35:54.575 INFO:tasks.workunit.client.0.smithi123.stdout:Skipping due to unmet REQUIRE 2024-09-16T17:35:54.575 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.ObjectPosition (154 ms) 2024-09-16T17:35:54.633 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] 16 tests from TestJournalReplay (5191 ms total) 2024-09-16T17:35:54.633 INFO:tasks.workunit.client.0.smithi123.stdout: 2024-09-16T17:35:54.633 INFO:tasks.workunit.client.0.smithi123.stdout:[----------] Global test environment tear-down 2024-09-16T17:35:54.633 INFO:tasks.workunit.client.0.smithi123.stdout:[==========] 16 tests from 1 test suite ran. (7197 ms total) 2024-09-16T17:35:54.633 INFO:tasks.workunit.client.0.smithi123.stdout:[ PASSED ] 0 tests. 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] 16 tests, listed below: 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.AioDiscardEvent 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.AioWriteEvent 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.AioFlushEvent 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapCreate 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapProtect 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapUnprotect 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapRename 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapRollback 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.SnapRemove 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.Rename 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.Resize 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.Flatten 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.UpdateFeatures 2024-09-16T17:35:54.634 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.MetadataSet 2024-09-16T17:35:54.635 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.MetadataRemove 2024-09-16T17:35:54.635 INFO:tasks.workunit.client.0.smithi123.stdout:[ SKIPPED ] TestJournalReplay.ObjectPosition 2024-09-16T17:35:54.641 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T17:35:54.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T17:35:54.673 INFO:tasks.workunit:Stopping ['rbd/crimson/test_crimson_librbd.sh'] on client.0... 2024-09-16T17:35:54.673 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T17:35:55.187 DEBUG:teuthology.parallel:result is None 2024-09-16T17:35:55.187 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T17:35:55.216 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T17:35:55.216 DEBUG:teuthology.orchestra.run.smithi123:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-16T17:35:55.286 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T17:35:55.287 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T17:35:55.296 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T17:35:55.296 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T17:35:55.523 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:55.515+0000 7fd1b7b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:55.530 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:55.529+0000 7fd1b7b91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:55.754 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T17:35:55.754 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T17:35:55.770 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":2178,"stamp":"2024-09-16T17:35:55.258383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":20528,"kb_used_data":20528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156872656,"statfs":{"total":160658620416,"available":160637599744,"internally_reserved":0,"allocated":21020672,"data_stored":21020672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878186"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.522705+0000","last_change":"2024-09-16T16:46:00.635813+0000","last_active":"2024-09-16T17:35:51.522705+0000","last_peered":"2024-09-16T17:35:51.522705+0000","last_clean":"2024-09-16T17:35:51.522705+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T17:35:51.522705+0000","last_undegraded":"2024-09-16T17:35:51.522705+0000","last_fullsized":"2024-09-16T17:35:51.522705+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.531461+0000","last_change":"2024-09-16T16:46:00.639079+0000","last_active":"2024-09-16T17:35:51.531461+0000","last_peered":"2024-09-16T17:35:51.531461+0000","last_clean":"2024-09-16T17:35:51.531461+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T17:35:51.531461+0000","last_undegraded":"2024-09-16T17:35:51.531461+0000","last_fullsized":"2024-09-16T17:35:51.531461+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526067+0000","last_change":"2024-09-16T16:46:00.635095+0000","last_active":"2024-09-16T17:35:51.526067+0000","last_peered":"2024-09-16T17:35:51.526067+0000","last_clean":"2024-09-16T17:35:51.526067+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T17:35:51.526067+0000","last_undegraded":"2024-09-16T17:35:51.526067+0000","last_fullsized":"2024-09-16T17:35:51.526067+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526551+0000","last_change":"2024-09-16T16:46:00.635505+0000","last_active":"2024-09-16T17:35:51.526551+0000","last_peered":"2024-09-16T17:35:51.526551+0000","last_clean":"2024-09-16T17:35:51.526551+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T17:35:51.526551+0000","last_undegraded":"2024-09-16T17:35:51.526551+0000","last_fullsized":"2024-09-16T17:35:51.526551+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":2020,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.613798+0000","last_change":"2024-09-16T16:46:00.637335+0000","last_active":"2024-09-16T17:35:51.613798+0000","last_peered":"2024-09-16T17:35:51.613798+0000","last_clean":"2024-09-16T17:35:51.613798+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T17:35:51.613798+0000","last_undegraded":"2024-09-16T17:35:51.613798+0000","last_fullsized":"2024-09-16T17:35:51.613798+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1949,"reported_epoch":660,"state":"active+clean","last_fresh":"2024-09-16T17:35:25.445927+0000","last_change":"2024-09-16T16:46:00.640092+0000","last_active":"2024-09-16T17:35:25.445927+0000","last_peered":"2024-09-16T17:35:25.445927+0000","last_clean":"2024-09-16T17:35:25.445927+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T17:35:25.445927+0000","last_undegraded":"2024-09-16T17:35:25.445927+0000","last_fullsized":"2024-09-16T17:35:25.445927+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1949,"reported_epoch":660,"state":"active+clean","last_fresh":"2024-09-16T17:35:25.422537+0000","last_change":"2024-09-16T16:46:00.634304+0000","last_active":"2024-09-16T17:35:25.422537+0000","last_peered":"2024-09-16T17:35:25.422537+0000","last_clean":"2024-09-16T17:35:25.422537+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T17:35:25.422537+0000","last_undegraded":"2024-09-16T17:35:25.422537+0000","last_fullsized":"2024-09-16T17:35:25.422537+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":2023,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.540993+0000","last_change":"2024-09-16T16:46:00.638883+0000","last_active":"2024-09-16T17:35:51.540993+0000","last_peered":"2024-09-16T17:35:51.540993+0000","last_clean":"2024-09-16T17:35:51.540993+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T17:35:51.540993+0000","last_undegraded":"2024-09-16T17:35:51.540993+0000","last_fullsized":"2024-09-16T17:35:51.540993+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":2091,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.530641+0000","last_change":"2024-09-16T16:45:57.684651+0000","last_active":"2024-09-16T17:35:51.530641+0000","last_peered":"2024-09-16T17:35:51.530641+0000","last_clean":"2024-09-16T17:35:51.530641+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T17:35:51.530641+0000","last_undegraded":"2024-09-16T17:35:51.530641+0000","last_fullsized":"2024-09-16T17:35:51.530641+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160636231680,"internally_reserved":0,"allocated":22388736,"data_stored":22388736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654708626,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4336,"kb_used_data":4336,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293392,"statfs":{"total":53552873472,"available":53548433408,"internally_reserved":0,"allocated":4440064,"data_stored":4440064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359741358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7784,"kb_used_data":7784,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289944,"statfs":{"total":53552873472,"available":53544902656,"internally_reserved":0,"allocated":7970816,"data_stored":7970816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359741356,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8408,"kb_used_data":8408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289320,"statfs":{"total":53552873472,"available":53544263680,"internally_reserved":0,"allocated":8609792,"data_stored":8609792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53544079360,"internally_reserved":0,"allocated":8794112,"data_stored":8794112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53548093440,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T17:35:55.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T17:35:55.895 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:55.889+0000 7fc033084640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:55.900 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:55.899+0000 7fc033084640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:56.099 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T17:35:56.099 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T17:35:56.114 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":2178,"stamp":"2024-09-16T17:35:55.258383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":20528,"kb_used_data":20528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156872656,"statfs":{"total":160658620416,"available":160637599744,"internally_reserved":0,"allocated":21020672,"data_stored":21020672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878186"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.522705+0000","last_change":"2024-09-16T16:46:00.635813+0000","last_active":"2024-09-16T17:35:51.522705+0000","last_peered":"2024-09-16T17:35:51.522705+0000","last_clean":"2024-09-16T17:35:51.522705+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T17:35:51.522705+0000","last_undegraded":"2024-09-16T17:35:51.522705+0000","last_fullsized":"2024-09-16T17:35:51.522705+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.531461+0000","last_change":"2024-09-16T16:46:00.639079+0000","last_active":"2024-09-16T17:35:51.531461+0000","last_peered":"2024-09-16T17:35:51.531461+0000","last_clean":"2024-09-16T17:35:51.531461+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T17:35:51.531461+0000","last_undegraded":"2024-09-16T17:35:51.531461+0000","last_fullsized":"2024-09-16T17:35:51.531461+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526067+0000","last_change":"2024-09-16T16:46:00.635095+0000","last_active":"2024-09-16T17:35:51.526067+0000","last_peered":"2024-09-16T17:35:51.526067+0000","last_clean":"2024-09-16T17:35:51.526067+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T17:35:51.526067+0000","last_undegraded":"2024-09-16T17:35:51.526067+0000","last_fullsized":"2024-09-16T17:35:51.526067+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526551+0000","last_change":"2024-09-16T16:46:00.635505+0000","last_active":"2024-09-16T17:35:51.526551+0000","last_peered":"2024-09-16T17:35:51.526551+0000","last_clean":"2024-09-16T17:35:51.526551+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T17:35:51.526551+0000","last_undegraded":"2024-09-16T17:35:51.526551+0000","last_fullsized":"2024-09-16T17:35:51.526551+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":2020,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.613798+0000","last_change":"2024-09-16T16:46:00.637335+0000","last_active":"2024-09-16T17:35:51.613798+0000","last_peered":"2024-09-16T17:35:51.613798+0000","last_clean":"2024-09-16T17:35:51.613798+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T17:35:51.613798+0000","last_undegraded":"2024-09-16T17:35:51.613798+0000","last_fullsized":"2024-09-16T17:35:51.613798+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1949,"reported_epoch":660,"state":"active+clean","last_fresh":"2024-09-16T17:35:25.445927+0000","last_change":"2024-09-16T16:46:00.640092+0000","last_active":"2024-09-16T17:35:25.445927+0000","last_peered":"2024-09-16T17:35:25.445927+0000","last_clean":"2024-09-16T17:35:25.445927+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T17:35:25.445927+0000","last_undegraded":"2024-09-16T17:35:25.445927+0000","last_fullsized":"2024-09-16T17:35:25.445927+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1949,"reported_epoch":660,"state":"active+clean","last_fresh":"2024-09-16T17:35:25.422537+0000","last_change":"2024-09-16T16:46:00.634304+0000","last_active":"2024-09-16T17:35:25.422537+0000","last_peered":"2024-09-16T17:35:25.422537+0000","last_clean":"2024-09-16T17:35:25.422537+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T17:35:25.422537+0000","last_undegraded":"2024-09-16T17:35:25.422537+0000","last_fullsized":"2024-09-16T17:35:25.422537+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":2023,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.540993+0000","last_change":"2024-09-16T16:46:00.638883+0000","last_active":"2024-09-16T17:35:51.540993+0000","last_peered":"2024-09-16T17:35:51.540993+0000","last_clean":"2024-09-16T17:35:51.540993+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T17:35:51.540993+0000","last_undegraded":"2024-09-16T17:35:51.540993+0000","last_fullsized":"2024-09-16T17:35:51.540993+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":2091,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.530641+0000","last_change":"2024-09-16T16:45:57.684651+0000","last_active":"2024-09-16T17:35:51.530641+0000","last_peered":"2024-09-16T17:35:51.530641+0000","last_clean":"2024-09-16T17:35:51.530641+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T17:35:51.530641+0000","last_undegraded":"2024-09-16T17:35:51.530641+0000","last_fullsized":"2024-09-16T17:35:51.530641+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160636231680,"internally_reserved":0,"allocated":22388736,"data_stored":22388736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654708626,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4336,"kb_used_data":4336,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293392,"statfs":{"total":53552873472,"available":53548433408,"internally_reserved":0,"allocated":4440064,"data_stored":4440064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359741358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7784,"kb_used_data":7784,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289944,"statfs":{"total":53552873472,"available":53544902656,"internally_reserved":0,"allocated":7970816,"data_stored":7970816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359741356,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8408,"kb_used_data":8408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289320,"statfs":{"total":53552873472,"available":53544263680,"internally_reserved":0,"allocated":8609792,"data_stored":8609792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53544079360,"internally_reserved":0,"allocated":8794112,"data_stored":8794112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53548093440,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T17:35:56.115 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T17:35:56.115 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T17:35:56.228 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:56.223+0000 7f35cf311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:56.232 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:56.232+0000 7f35cf311640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:56.427 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T17:35:56.427 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T17:35:56.442 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":2178,"stamp":"2024-09-16T17:35:55.258383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":20528,"kb_used_data":20528,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156872656,"statfs":{"total":160658620416,"available":160637599744,"internally_reserved":0,"allocated":21020672,"data_stored":21020672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.878186"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.522705+0000","last_change":"2024-09-16T16:46:00.635813+0000","last_active":"2024-09-16T17:35:51.522705+0000","last_peered":"2024-09-16T17:35:51.522705+0000","last_clean":"2024-09-16T17:35:51.522705+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T17:35:51.522705+0000","last_undegraded":"2024-09-16T17:35:51.522705+0000","last_fullsized":"2024-09-16T17:35:51.522705+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.531461+0000","last_change":"2024-09-16T16:46:00.639079+0000","last_active":"2024-09-16T17:35:51.531461+0000","last_peered":"2024-09-16T17:35:51.531461+0000","last_clean":"2024-09-16T17:35:51.531461+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T17:35:51.531461+0000","last_undegraded":"2024-09-16T17:35:51.531461+0000","last_fullsized":"2024-09-16T17:35:51.531461+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526067+0000","last_change":"2024-09-16T16:46:00.635095+0000","last_active":"2024-09-16T17:35:51.526067+0000","last_peered":"2024-09-16T17:35:51.526067+0000","last_clean":"2024-09-16T17:35:51.526067+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T17:35:51.526067+0000","last_undegraded":"2024-09-16T17:35:51.526067+0000","last_fullsized":"2024-09-16T17:35:51.526067+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526551+0000","last_change":"2024-09-16T16:46:00.635505+0000","last_active":"2024-09-16T17:35:51.526551+0000","last_peered":"2024-09-16T17:35:51.526551+0000","last_clean":"2024-09-16T17:35:51.526551+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T17:35:51.526551+0000","last_undegraded":"2024-09-16T17:35:51.526551+0000","last_fullsized":"2024-09-16T17:35:51.526551+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":2020,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.613798+0000","last_change":"2024-09-16T16:46:00.637335+0000","last_active":"2024-09-16T17:35:51.613798+0000","last_peered":"2024-09-16T17:35:51.613798+0000","last_clean":"2024-09-16T17:35:51.613798+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T17:35:51.613798+0000","last_undegraded":"2024-09-16T17:35:51.613798+0000","last_fullsized":"2024-09-16T17:35:51.613798+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1949,"reported_epoch":660,"state":"active+clean","last_fresh":"2024-09-16T17:35:25.445927+0000","last_change":"2024-09-16T16:46:00.640092+0000","last_active":"2024-09-16T17:35:25.445927+0000","last_peered":"2024-09-16T17:35:25.445927+0000","last_clean":"2024-09-16T17:35:25.445927+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T17:35:25.445927+0000","last_undegraded":"2024-09-16T17:35:25.445927+0000","last_fullsized":"2024-09-16T17:35:25.445927+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1949,"reported_epoch":660,"state":"active+clean","last_fresh":"2024-09-16T17:35:25.422537+0000","last_change":"2024-09-16T16:46:00.634304+0000","last_active":"2024-09-16T17:35:25.422537+0000","last_peered":"2024-09-16T17:35:25.422537+0000","last_clean":"2024-09-16T17:35:25.422537+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T17:35:25.422537+0000","last_undegraded":"2024-09-16T17:35:25.422537+0000","last_fullsized":"2024-09-16T17:35:25.422537+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":2023,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.540993+0000","last_change":"2024-09-16T16:46:00.638883+0000","last_active":"2024-09-16T17:35:51.540993+0000","last_peered":"2024-09-16T17:35:51.540993+0000","last_clean":"2024-09-16T17:35:51.540993+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T17:35:51.540993+0000","last_undegraded":"2024-09-16T17:35:51.540993+0000","last_fullsized":"2024-09-16T17:35:51.540993+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":2091,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.530641+0000","last_change":"2024-09-16T16:45:57.684651+0000","last_active":"2024-09-16T17:35:51.530641+0000","last_peered":"2024-09-16T17:35:51.530641+0000","last_clean":"2024-09-16T17:35:51.530641+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T17:35:51.530641+0000","last_undegraded":"2024-09-16T17:35:51.530641+0000","last_fullsized":"2024-09-16T17:35:51.530641+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160636231680,"internally_reserved":0,"allocated":22388736,"data_stored":22388736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654708626,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4336,"kb_used_data":4336,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293392,"statfs":{"total":53552873472,"available":53548433408,"internally_reserved":0,"allocated":4440064,"data_stored":4440064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359741358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7784,"kb_used_data":7784,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289944,"statfs":{"total":53552873472,"available":53544902656,"internally_reserved":0,"allocated":7970816,"data_stored":7970816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359741356,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8408,"kb_used_data":8408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289320,"statfs":{"total":53552873472,"available":53544263680,"internally_reserved":0,"allocated":8609792,"data_stored":8609792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53544079360,"internally_reserved":0,"allocated":8794112,"data_stored":8794112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53548093440,"internally_reserved":0,"allocated":4780032,"data_stored":4780032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T17:35:56.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T17:35:56.557 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:56.552+0000 7ff753b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:56.561 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:56.561+0000 7ff753b11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:56.865 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T17:35:56.865 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":684,"fsid":"0d2cd943-ff49-4927-aa46-a373e0b8c2c2","created":"2024-09-16T16:45:47.672610+0000","modified":"2024-09-16T17:35:54.627075+0000","last_up_change":"2024-09-16T16:45:55.570159+0000","last_in_change":"2024-09-16T16:45:49.550536+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":45,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T16:45:56.535754+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T16:45:57.501018+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"053aba5d-d94f-4e73-94b4-75158ee9a96b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":680,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":1432985695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6803","nonce":1432985695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6811","nonce":1432985695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":1432985695}]},"public_addr":"172.21.15.123:6802/1432985695","cluster_addr":"172.21.15.123:6803/1432985695","heartbeat_back_addr":"172.21.15.123:6811/1432985695","heartbeat_front_addr":"172.21.15.123:6812/1432985695","state":["exists","up"]},{"osd":1,"uuid":"d2f0256b-4436-4985-82c9-69a8dbc8d9c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":680,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6809","nonce":1330500426}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":1330500426}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6805","nonce":1330500426}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":1330500426}]},"public_addr":"172.21.15.123:6809/1330500426","cluster_addr":"172.21.15.123:6808/1330500426","heartbeat_back_addr":"172.21.15.123:6805/1330500426","heartbeat_front_addr":"172.21.15.123:6806/1330500426","state":["exists","up"]},{"osd":2,"uuid":"c117682f-533a-498e-8b07-0dd88ef17d4a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":680,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":1440730160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6807","nonce":1440730160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6813","nonce":1440730160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":1440730160}]},"public_addr":"172.21.15.123:6804/1440730160","cluster_addr":"172.21.15.123:6807/1440730160","heartbeat_back_addr":"172.21.15.123:6813/1440730160","heartbeat_front_addr":"172.21.15.123:6810/1440730160","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]},{"pool":3,"snaps":[{"begin":2,"length":3},{"begin":7,"length":11},{"begin":23,"length":4},{"begin":37,"length":1},{"begin":41,"length":1},{"begin":44,"length":2},{"begin":69,"length":1},{"begin":71,"length":1}]},{"pool":4,"snaps":[{"begin":2,"length":1}]},{"pool":5,"snaps":[{"begin":2,"length":1}]},{"pool":6,"snaps":[{"begin":2,"length":1}]},{"pool":7,"snaps":[{"begin":2,"length":1}]},{"pool":8,"snaps":[{"begin":2,"length":1}]},{"pool":9,"snaps":[{"begin":2,"length":1}]},{"pool":10,"snaps":[{"begin":2,"length":4}]},{"pool":11,"snaps":[{"begin":2,"length":1}]},{"pool":12,"snaps":[{"begin":2,"length":1}]},{"pool":13,"snaps":[{"begin":2,"length":1}]},{"pool":14,"snaps":[{"begin":2,"length":1}]},{"pool":16,"snaps":[{"begin":2,"length":1}]},{"pool":17,"snaps":[{"begin":2,"length":1}]},{"pool":18,"snaps":[{"begin":2,"length":2}]},{"pool":19,"snaps":[{"begin":2,"length":1}]},{"pool":20,"snaps":[{"begin":2,"length":1}]},{"pool":21,"snaps":[{"begin":2,"length":1}]},{"pool":22,"snaps":[{"begin":2,"length":3}]},{"pool":23,"snaps":[{"begin":2,"length":9}]},{"pool":24,"snaps":[{"begin":2,"length":1}]},{"pool":25,"snaps":[{"begin":2,"length":1}]},{"pool":26,"snaps":[{"begin":2,"length":1}]},{"pool":27,"snaps":[{"begin":2,"length":1}]},{"pool":28,"snaps":[{"begin":2,"length":1}]},{"pool":29,"snaps":[{"begin":2,"length":1}]},{"pool":30,"snaps":[{"begin":2,"length":1}]},{"pool":31,"snaps":[{"begin":2,"length":1}]},{"pool":32,"snaps":[{"begin":2,"length":1}]},{"pool":33,"snaps":[{"begin":2,"length":7},{"begin":14,"length":2},{"begin":19,"length":1}]},{"pool":34,"snaps":[{"begin":2,"length":46}]},{"pool":35,"snaps":[{"begin":2,"length":13}]},{"pool":38,"snaps":[{"begin":2,"length":3}]},{"pool":39,"snaps":[{"begin":2,"length":3}]},{"pool":40,"snaps":[{"begin":2,"length":1}]},{"pool":41,"snaps":[{"begin":2,"length":1}]},{"pool":42,"snaps":[{"begin":2,"length":1}]},{"pool":43,"snaps":[{"begin":2,"length":1}]},{"pool":44,"snaps":[{"begin":2,"length":1}]},{"pool":45,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T17:35:57.881 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T17:35:57.882 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T17:35:57.999 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:57.989+0000 7f8b778ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:58.003 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:58.002+0000 7f8b778ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:58.017 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-16T17:35:58.017 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "" 2024-09-16T17:35:58.017 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-16T17:35:58.029 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T17:35:58.140 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:58.134+0000 7f299f488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:58.144 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:58.144+0000 7f299f488640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:58.508 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T17:35:58.530 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T17:35:58.530 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T17:35:58.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:58.637+0000 7f8e00e9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:58.650 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:58.649+0000 7f8e00e9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:58.694 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-16T17:35:58.694 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "" 2024-09-16T17:35:58.694 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-16T17:35:58.710 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T17:35:58.822 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:58.816+0000 7fcbf3351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:58.826 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:58.825+0000 7fcbf3351640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:59.018 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T17:35:59.037 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T17:35:59.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T17:35:59.162 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:59.152+0000 7f2c812bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:59.170 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:59.169+0000 7f2c812bd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:59.185 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-16T17:35:59.185 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "" 2024-09-16T17:35:59.185 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-16T17:35:59.197 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T17:35:59.309 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:59.304+0000 7f78e4391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:59.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:59.313+0000 7f78e4391640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:59.509 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T17:35:59.528 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T17:35:59.642 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:59.638+0000 7ff7eb6a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:59.646 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:35:59.646+0000 7ff7eb6a3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:35:59.836 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T17:35:59.836 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T17:35:59.849 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":2181,"stamp":"2024-09-16T17:35:59.259179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":23984,"kb_used_data":23984,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156869200,"statfs":{"total":160658620416,"available":160634060800,"internally_reserved":0,"allocated":24559616,"data_stored":24559616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.867435"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.522705+0000","last_change":"2024-09-16T16:46:00.635813+0000","last_active":"2024-09-16T17:35:51.522705+0000","last_peered":"2024-09-16T17:35:51.522705+0000","last_clean":"2024-09-16T17:35:51.522705+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T17:35:51.522705+0000","last_undegraded":"2024-09-16T17:35:51.522705+0000","last_fullsized":"2024-09-16T17:35:51.522705+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.531461+0000","last_change":"2024-09-16T16:46:00.639079+0000","last_active":"2024-09-16T17:35:51.531461+0000","last_peered":"2024-09-16T17:35:51.531461+0000","last_clean":"2024-09-16T17:35:51.531461+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T17:35:51.531461+0000","last_undegraded":"2024-09-16T17:35:51.531461+0000","last_fullsized":"2024-09-16T17:35:51.531461+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526067+0000","last_change":"2024-09-16T16:46:00.635095+0000","last_active":"2024-09-16T17:35:51.526067+0000","last_peered":"2024-09-16T17:35:51.526067+0000","last_clean":"2024-09-16T17:35:51.526067+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T17:35:51.526067+0000","last_undegraded":"2024-09-16T17:35:51.526067+0000","last_fullsized":"2024-09-16T17:35:51.526067+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526551+0000","last_change":"2024-09-16T16:46:00.635505+0000","last_active":"2024-09-16T17:35:51.526551+0000","last_peered":"2024-09-16T17:35:51.526551+0000","last_clean":"2024-09-16T17:35:51.526551+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T17:35:51.526551+0000","last_undegraded":"2024-09-16T17:35:51.526551+0000","last_fullsized":"2024-09-16T17:35:51.526551+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":2020,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.613798+0000","last_change":"2024-09-16T16:46:00.637335+0000","last_active":"2024-09-16T17:35:51.613798+0000","last_peered":"2024-09-16T17:35:51.613798+0000","last_clean":"2024-09-16T17:35:51.613798+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T17:35:51.613798+0000","last_undegraded":"2024-09-16T17:35:51.613798+0000","last_fullsized":"2024-09-16T17:35:51.613798+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2021,"reported_epoch":684,"state":"active+clean","last_fresh":"2024-09-16T17:35:54.872415+0000","last_change":"2024-09-16T16:46:00.640092+0000","last_active":"2024-09-16T17:35:54.872415+0000","last_peered":"2024-09-16T17:35:54.872415+0000","last_clean":"2024-09-16T17:35:54.872415+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T17:35:54.872415+0000","last_undegraded":"2024-09-16T17:35:54.872415+0000","last_fullsized":"2024-09-16T17:35:54.872415+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2021,"reported_epoch":684,"state":"active+clean","last_fresh":"2024-09-16T17:35:54.812245+0000","last_change":"2024-09-16T16:46:00.634304+0000","last_active":"2024-09-16T17:35:54.812245+0000","last_peered":"2024-09-16T17:35:54.812245+0000","last_clean":"2024-09-16T17:35:54.812245+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T17:35:54.812245+0000","last_undegraded":"2024-09-16T17:35:54.812245+0000","last_fullsized":"2024-09-16T17:35:54.812245+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":2023,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.540993+0000","last_change":"2024-09-16T16:46:00.638883+0000","last_active":"2024-09-16T17:35:51.540993+0000","last_peered":"2024-09-16T17:35:51.540993+0000","last_clean":"2024-09-16T17:35:51.540993+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T17:35:51.540993+0000","last_undegraded":"2024-09-16T17:35:51.540993+0000","last_fullsized":"2024-09-16T17:35:51.540993+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":2091,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.530641+0000","last_change":"2024-09-16T16:45:57.684651+0000","last_active":"2024-09-16T17:35:51.530641+0000","last_peered":"2024-09-16T17:35:51.530641+0000","last_clean":"2024-09-16T17:35:51.530641+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T17:35:51.530641+0000","last_undegraded":"2024-09-16T17:35:51.530641+0000","last_fullsized":"2024-09-16T17:35:51.530641+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160636477440,"internally_reserved":0,"allocated":22142976,"data_stored":22142976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654708656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7792,"kb_used_data":7792,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289936,"statfs":{"total":53552873472,"available":53544894464,"internally_reserved":0,"allocated":7979008,"data_stored":7979008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359741358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7784,"kb_used_data":7784,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289944,"statfs":{"total":53552873472,"available":53544902656,"internally_reserved":0,"allocated":7970816,"data_stored":7970816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359741356,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8408,"kb_used_data":8408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289320,"statfs":{"total":53552873472,"available":53544263680,"internally_reserved":0,"allocated":8609792,"data_stored":8609792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53544079360,"internally_reserved":0,"allocated":8794112,"data_stored":8794112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53548339200,"internally_reserved":0,"allocated":4534272,"data_stored":4534272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.853 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T16:45:56.579963+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:35:59.854 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T17:36:19.855 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T17:36:19.976 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:36:19.971+0000 7f2f2cd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:36:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:36:19.979+0000 7f2f2cd91640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:36:20.161 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T17:36:20.162 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T17:36:20.175 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":2191,"stamp":"2024-09-16T17:36:19.263429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":23984,"kb_used_data":23984,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156869200,"statfs":{"total":160658620416,"available":160634060800,"internally_reserved":0,"allocated":24559616,"data_stored":24559616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002753"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.522705+0000","last_change":"2024-09-16T16:46:00.635813+0000","last_active":"2024-09-16T17:35:51.522705+0000","last_peered":"2024-09-16T17:35:51.522705+0000","last_clean":"2024-09-16T17:35:51.522705+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T17:35:51.522705+0000","last_undegraded":"2024-09-16T17:35:51.522705+0000","last_fullsized":"2024-09-16T17:35:51.522705+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.531461+0000","last_change":"2024-09-16T16:46:00.639079+0000","last_active":"2024-09-16T17:35:51.531461+0000","last_peered":"2024-09-16T17:35:51.531461+0000","last_clean":"2024-09-16T17:35:51.531461+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T17:35:51.531461+0000","last_undegraded":"2024-09-16T17:35:51.531461+0000","last_fullsized":"2024-09-16T17:35:51.531461+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526067+0000","last_change":"2024-09-16T16:46:00.635095+0000","last_active":"2024-09-16T17:35:51.526067+0000","last_peered":"2024-09-16T17:35:51.526067+0000","last_clean":"2024-09-16T17:35:51.526067+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T17:35:51.526067+0000","last_undegraded":"2024-09-16T17:35:51.526067+0000","last_fullsized":"2024-09-16T17:35:51.526067+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2022,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.526551+0000","last_change":"2024-09-16T16:46:00.635505+0000","last_active":"2024-09-16T17:35:51.526551+0000","last_peered":"2024-09-16T17:35:51.526551+0000","last_clean":"2024-09-16T17:35:51.526551+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T17:35:51.526551+0000","last_undegraded":"2024-09-16T17:35:51.526551+0000","last_fullsized":"2024-09-16T17:35:51.526551+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":2020,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.613798+0000","last_change":"2024-09-16T16:46:00.637335+0000","last_active":"2024-09-16T17:35:51.613798+0000","last_peered":"2024-09-16T17:35:51.613798+0000","last_clean":"2024-09-16T17:35:51.613798+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T17:35:51.613798+0000","last_undegraded":"2024-09-16T17:35:51.613798+0000","last_fullsized":"2024-09-16T17:35:51.613798+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2021,"reported_epoch":684,"state":"active+clean","last_fresh":"2024-09-16T17:35:54.872415+0000","last_change":"2024-09-16T16:46:00.640092+0000","last_active":"2024-09-16T17:35:54.872415+0000","last_peered":"2024-09-16T17:35:54.872415+0000","last_clean":"2024-09-16T17:35:54.872415+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T17:35:54.872415+0000","last_undegraded":"2024-09-16T17:35:54.872415+0000","last_fullsized":"2024-09-16T17:35:54.872415+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2021,"reported_epoch":684,"state":"active+clean","last_fresh":"2024-09-16T17:35:54.812245+0000","last_change":"2024-09-16T16:46:00.634304+0000","last_active":"2024-09-16T17:35:54.812245+0000","last_peered":"2024-09-16T17:35:54.812245+0000","last_clean":"2024-09-16T17:35:54.812245+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T17:35:54.812245+0000","last_undegraded":"2024-09-16T17:35:54.812245+0000","last_fullsized":"2024-09-16T17:35:54.812245+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":2023,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.540993+0000","last_change":"2024-09-16T16:46:00.638883+0000","last_active":"2024-09-16T17:35:51.540993+0000","last_peered":"2024-09-16T17:35:51.540993+0000","last_clean":"2024-09-16T17:35:51.540993+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T17:35:51.540993+0000","last_undegraded":"2024-09-16T17:35:51.540993+0000","last_fullsized":"2024-09-16T17:35:51.540993+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:57.584377+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":2091,"reported_epoch":683,"state":"active+clean","last_fresh":"2024-09-16T17:35:51.530641+0000","last_change":"2024-09-16T16:45:57.684651+0000","last_active":"2024-09-16T17:35:51.530641+0000","last_peered":"2024-09-16T17:35:51.530641+0000","last_clean":"2024-09-16T17:35:51.530641+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T17:35:51.530641+0000","last_undegraded":"2024-09-16T17:35:51.530641+0000","last_fullsized":"2024-09-16T17:35:51.530641+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T16:45:56.579963+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160636477440,"internally_reserved":0,"allocated":22142976,"data_stored":22142976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654708656,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7792,"kb_used_data":7792,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289936,"statfs":{"total":53552873472,"available":53544894464,"internally_reserved":0,"allocated":7979008,"data_stored":7979008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359741358,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":7784,"kb_used_data":7784,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289944,"statfs":{"total":53552873472,"available":53544902656,"internally_reserved":0,"allocated":7970816,"data_stored":7970816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359741356,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":8408,"kb_used_data":8408,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52289320,"statfs":{"total":53552873472,"available":53544263680,"internally_reserved":0,"allocated":8609792,"data_stored":8609792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53544079360,"internally_reserved":0,"allocated":8794112,"data_stored":8794112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53544058880,"internally_reserved":0,"allocated":8814592,"data_stored":8814592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53548339200,"internally_reserved":0,"allocated":4534272,"data_stored":4534272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T17:36:20.176 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.176 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.176 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.176 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.176 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.177 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.177 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.177 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T16:45:57.584377+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=57, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.177 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T16:45:56.579963+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=16, tm_min=45, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=17, tm_min=35, tm_sec=56, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T17:36:20.177 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T17:36:40.178 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T17:36:40.299 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:36:40.294+0000 7fb07c846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:36:40.303 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:36:40.302+0000 7fb07c846640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:36:40.488 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-16T17:36:40.488 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2024-09-16T17:36:40.502 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":2201,"stamp":"2024-09-16T17:36:39.268362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156893184,"kb_used":11384,"kb_used_data":11384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156881800,"statfs":{"total":160658620416,"available":160646963200,"internally_reserved":0,"allocated":11657216,"data_stored":11657216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002933"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":2030,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.027283+0000","last_change":"2024-09-16T17:35:59.072882+0000","last_active":"2024-09-16T17:35:59.027283+0000","last_peered":"2024-09-16T17:35:59.027283+0000","last_clean":"2024-09-16T17:35:59.027283+0000","last_became_active":"2024-09-16T16:45:58.826427+0000","last_became_peered":"2024-09-16T16:45:58.826427+0000","last_unstale":"2024-09-16T17:35:59.027283+0000","last_undegraded":"2024-09-16T17:35:59.027283+0000","last_fullsized":"2024-09-16T17:35:59.027283+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T17:35:59.072681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.072681+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2030,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.027044+0000","last_change":"2024-09-16T17:35:59.050016+0000","last_active":"2024-09-16T17:35:59.027044+0000","last_peered":"2024-09-16T17:35:59.027044+0000","last_clean":"2024-09-16T17:35:59.027044+0000","last_became_active":"2024-09-16T16:45:58.837990+0000","last_became_peered":"2024-09-16T16:45:58.837990+0000","last_unstale":"2024-09-16T17:35:59.027044+0000","last_undegraded":"2024-09-16T17:35:59.027044+0000","last_fullsized":"2024-09-16T17:35:59.027044+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T17:35:59.049776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.049776+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2030,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.024957+0000","last_change":"2024-09-16T17:35:59.069102+0000","last_active":"2024-09-16T17:35:59.024957+0000","last_peered":"2024-09-16T17:35:59.024957+0000","last_clean":"2024-09-16T17:35:59.024957+0000","last_became_active":"2024-09-16T16:45:58.780300+0000","last_became_peered":"2024-09-16T16:45:58.780300+0000","last_unstale":"2024-09-16T17:35:59.024957+0000","last_undegraded":"2024-09-16T17:35:59.024957+0000","last_fullsized":"2024-09-16T17:35:59.024957+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T17:35:59.068934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.068934+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2030,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.024005+0000","last_change":"2024-09-16T17:35:59.076063+0000","last_active":"2024-09-16T17:35:59.024005+0000","last_peered":"2024-09-16T17:35:59.024005+0000","last_clean":"2024-09-16T17:35:59.024005+0000","last_became_active":"2024-09-16T16:45:58.788759+0000","last_became_peered":"2024-09-16T16:45:58.788759+0000","last_unstale":"2024-09-16T17:35:59.024005+0000","last_undegraded":"2024-09-16T17:35:59.024005+0000","last_fullsized":"2024-09-16T17:35:59.024005+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T17:35:59.075894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.075894+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":2028,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:58.509737+0000","last_change":"2024-09-16T17:35:58.544666+0000","last_active":"2024-09-16T17:35:58.509737+0000","last_peered":"2024-09-16T17:35:58.509737+0000","last_clean":"2024-09-16T17:35:58.509737+0000","last_became_active":"2024-09-16T16:45:58.777961+0000","last_became_peered":"2024-09-16T16:45:58.777961+0000","last_unstale":"2024-09-16T17:35:58.509737+0000","last_undegraded":"2024-09-16T17:35:58.509737+0000","last_fullsized":"2024-09-16T17:35:58.509737+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"14'2","last_scrub_stamp":"2024-09-16T17:35:58.544475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:58.544475+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2026,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.513469+0000","last_change":"2024-09-16T17:35:59.532162+0000","last_active":"2024-09-16T17:35:59.513469+0000","last_peered":"2024-09-16T17:35:59.513469+0000","last_clean":"2024-09-16T17:35:59.513469+0000","last_became_active":"2024-09-16T16:45:58.831770+0000","last_became_peered":"2024-09-16T16:45:58.831770+0000","last_unstale":"2024-09-16T17:35:59.513469+0000","last_undegraded":"2024-09-16T17:35:59.513469+0000","last_fullsized":"2024-09-16T17:35:59.513469+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T17:35:59.531986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.531986+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2026,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.511488+0000","last_change":"2024-09-16T17:35:59.530305+0000","last_active":"2024-09-16T17:35:59.511488+0000","last_peered":"2024-09-16T17:35:59.511488+0000","last_clean":"2024-09-16T17:35:59.511488+0000","last_became_active":"2024-09-16T16:45:58.791268+0000","last_became_peered":"2024-09-16T16:45:58.791268+0000","last_unstale":"2024-09-16T17:35:59.511488+0000","last_undegraded":"2024-09-16T17:35:59.511488+0000","last_fullsized":"2024-09-16T17:35:59.511488+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T17:35:59.530141+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.530141+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":2031,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.023070+0000","last_change":"2024-09-16T17:35:59.054282+0000","last_active":"2024-09-16T17:35:59.023070+0000","last_peered":"2024-09-16T17:35:59.023070+0000","last_clean":"2024-09-16T17:35:59.023070+0000","last_became_active":"2024-09-16T16:45:58.789365+0000","last_became_peered":"2024-09-16T16:45:58.789365+0000","last_unstale":"2024-09-16T17:35:59.023070+0000","last_undegraded":"2024-09-16T17:35:59.023070+0000","last_fullsized":"2024-09-16T17:35:59.023070+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'1","last_scrub_stamp":"2024-09-16T17:35:59.054100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.054100+0000","last_clean_scrub_stamp":"2024-09-16T16:45:57.584377+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'52","reported_seq":2099,"reported_epoch":685,"state":"active+clean","last_fresh":"2024-09-16T17:35:59.021249+0000","last_change":"2024-09-16T17:35:59.139448+0000","last_active":"2024-09-16T17:35:59.021249+0000","last_peered":"2024-09-16T17:35:59.021249+0000","last_clean":"2024-09-16T17:35:59.021249+0000","last_became_active":"2024-09-16T16:45:57.652868+0000","last_became_peered":"2024-09-16T16:45:57.652868+0000","last_unstale":"2024-09-16T17:35:59.021249+0000","last_undegraded":"2024-09-16T17:35:59.021249+0000","last_fullsized":"2024-09-16T17:35:59.021249+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"11'52","last_scrub_stamp":"2024-09-16T17:35:59.139251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T17:35:59.139251+0000","last_clean_scrub_stamp":"2024-09-16T16:45:56.579963+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53552873472,"available":53548552192,"internally_reserved":0,"allocated":4321280,"data_stored":4321280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160658620416,"available":160646963200,"internally_reserved":0,"allocated":11657216,"data_stored":11657216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654708686,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":3076,"kb_used_data":3076,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52294652,"statfs":{"total":53552873472,"available":53549723648,"internally_reserved":0,"allocated":3149824,"data_stored":3149824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359741388,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4220,"kb_used_data":4220,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293508,"statfs":{"total":53552873472,"available":53548552192,"internally_reserved":0,"allocated":4321280,"data_stored":4321280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359741386,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52297728,"kb_used":4088,"kb_used_data":4088,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52293640,"statfs":{"total":53552873472,"available":53548687360,"internally_reserved":0,"allocated":4186112,"data_stored":4186112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53552873472,"available":53548552192,"internally_reserved":0,"allocated":4321280,"data_stored":4321280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53552873472,"available":53548687360,"internally_reserved":0,"allocated":4186112,"data_stored":4186112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53552873472,"available":53548552192,"internally_reserved":0,"allocated":4321280,"data_stored":4321280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53552873472,"available":53549723648,"internally_reserved":0,"allocated":3149824,"data_stored":3149824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T17:36:40.503 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T17:36:40.610 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:36:40.602+0000 7f17bc511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:36:40.613 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-16T17:36:40.613+0000 7f17bc511640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2024-09-16T17:36:40.815 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T17:36:40.815 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T17:36:40.815 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T17:36:40.815 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T17:36:40.903 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T17:36:40.903 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T17:36:40.903 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T17:36:40.991 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T17:36:40.991 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T17:36:40.992 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T17:36:41.080 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T17:36:41.080 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T17:36:41.081 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T17:36:41.081 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T17:36:41.109 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T17:36:41.109 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T17:36:41.109 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T17:36:41.109 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T17:36:41.168 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T17:36:41.168 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T17:36:41.168 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'is full \(reached quota' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_FULL\)' | head -n 1 2024-09-16T17:36:41.238 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T17:36:41.238 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T17:36:41.427 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T17:36:41.427 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T17:36:41.541 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T17:36:41.541 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T17:36:41.606 INFO:tasks.ceph:Archiving mon data... 2024-09-16T17:36:41.607 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7907389/data/mon.a.tgz 2024-09-16T17:36:41.607 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-09-16T17:36:41.622 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.ZI2QZ1Vig8 2024-09-16T17:36:41.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ZI2QZ1Vig8 2024-09-16T17:36:41.754 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.ZI2QZ1Vig8 2024-09-16T17:36:41.837 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.ZI2QZ1Vig8 is 317KB 2024-09-16T17:36:41.931 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.ZI2QZ1Vig8 2024-09-16T17:36:41.946 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T17:36:41.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T17:36:42.065 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-16T17:36:42.065 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T17:36:42.066 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7907389/remote/smithi123/crash 2024-09-16T17:36:42.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T17:36:42.129 INFO:tasks.ceph:Compressing logs... 2024-09-16T17:36:42.130 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T17:36:42.218 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T17:36:42.218 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75225.log 2024-09-16T17:36:42.218 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75801.log 2024-09-16T17:36:42.218 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29232.log 2024-09-16T17:36:42.218 INFO:teuthology.orchestra.run.smithi123.stderr: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.75225.log.gz 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75717.log 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.75801.log.gz 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29104.log 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29232.log.gz 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29105.log 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.75717.log.gz 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29104.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29104.log.gz 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76009.log 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29105.log.gz 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29106.log 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.76009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75327.log 2024-09-16T17:36:42.219 INFO:teuthology.orchestra.run.smithi123.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.76009.log.gz 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58077.log 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29106.log.gz 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75327.log: 76.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.75327.log.gz 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75465.log 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.58077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28419.log 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.58077.log.gz 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75465.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.75465.log.gz 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28803.log 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45955.log 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28419.log.gz 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28803.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28803.log.gz 2024-09-16T17:36:42.220 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75549.log 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.45955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29360.log 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.45955.log.gz 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75549.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.75549.log.gz 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75278.log 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29620.log 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29360.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29360.log.gz 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29192.log 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.75278.log.gz 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29620.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75675.log 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29192.log.gz 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.audit.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.29620.log.gz 2024-09-16T17:36:42.221 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75927.log 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28982.log 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75675.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.75675.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75591.log 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.75927.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr: 88.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29318.log 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28982.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75885.log 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.75591.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29318.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29318.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27170.log 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39391.log 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.75885.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.tmp-client.admin.27170.log: 67.7%gzip -- replaced with /var/log/ceph/ceph.tmp-client.admin.27170.log.gz 2024-09-16T17:36:42.222 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75507.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29024.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr: 80.3% -- replaced with /var/log/ceph/ceph-client.admin.39391.log.gz 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75507.log: gzip -5 --verbose -- 49.6% /var/log/ceph/ceph-client.admin.28747.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75507.log.gz 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29024.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29024.log.gz 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28747.log: gzip -5 49.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.28747.log.gz -- 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr: /var/log/ceph/ceph-client.admin.68868.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28368.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.68868.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr: 80.4% -- replaced with /var/log/ceph/ceph-client.admin.68868.log.gz 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:gzip/var/log/ceph/ceph-client.admin.28368.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.28898.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28368.log.gz 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28828.log 2024-09-16T17:36:42.223 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75759.log 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28898.log.gz 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75175.log 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.admin.28828.log.gz 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28607.log 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.75759.log.gz 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75175.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.75175.log.gz 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74377.log 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28607.log.gz 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29231.log 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.74377.log: gzip 76.8% -- replaced with /var/log/ceph/ceph-client.admin.74377.log.gz 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29231.log: gzip 49.6% -- replaced with /var/log/ceph/ceph-client.admin.29231.log.gz 2024-09-16T17:36:42.224 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75633.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75123.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75633.log: gzip 48.8% -- replaced with /var/log/ceph/ceph-client.admin.75633.log.gz 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.75843.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46094.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75843.log: /var/log/ceph/ceph-client.admin.75123.log: 49.6% 82.9% -- replaced with /var/log/ceph/ceph-client.admin.75123.log.gz 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75843.log.gz 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75969.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28130.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75969.log: /var/log/ceph/ceph-client.admin.46094.log: 49.6% 80.4% -- replaced with /var/log/ceph/ceph-client.admin.46094.log.gz 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.75969.log.gz 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28218.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46023.log 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28130.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28130.log.gz 2024-09-16T17:36:42.225 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28218.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.28218.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28328.log 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.46023.log: 80.1% -- replaced with /var/log/ceph/ceph-client.admin.46023.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr: 88.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29402.log 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28328.log: gzip -5 --verbose 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28328.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr: -- /var/log/ceph/ceph-client.admin.52055.log 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.29402.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.29402.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39320.log 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.52055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28577.log 80.2% -- replaced with /var/log/ceph/ceph-client.admin.52055.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.39320.log: gzip 80.1% -- replaced with /var/log/ceph/ceph-client.admin.39320.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74502.log 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28577.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28577.log.gz 2024-09-16T17:36:42.226 INFO:teuthology.orchestra.run.smithi123.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74445.log 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.74502.log: gzip -5 78.6% -- replaced with /var/log/ceph/ceph-client.admin.74502.log.gz 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr: --verbose -- /var/log/ceph/ceph-client.admin.28940.log 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.74445.log: gzip 81.2% -5 --verbose -- /var/log/ceph/ceph-client.admin.75071.log 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74445.log.gz 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.28940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75007.log 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.28940.log.gz 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75071.log: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.75071.log.gz 2024-09-16T17:36:42.227 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ceph-client.admin.75007.log: 78.2% -- replaced with /var/log/ceph/ceph-client.admin.75007.log.gz 2024-09-16T17:36:42.470 INFO:teuthology.orchestra.run.smithi123.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T17:36:43.274 INFO:teuthology.orchestra.run.smithi123.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T17:36:52.252 INFO:teuthology.orchestra.run.smithi123.stderr: 95.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T17:36:54.041 INFO:teuthology.orchestra.run.smithi123.stderr: 95.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T17:36:54.936 INFO:teuthology.orchestra.run.smithi123.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T17:36:54.938 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-16T17:36:54.938 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m12.753s 2024-09-16T17:36:54.938 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m34.434s 2024-09-16T17:36:54.938 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m1.529s 2024-09-16T17:36:54.938 INFO:tasks.ceph:Archiving logs... 2024-09-16T17:36:54.938 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7907389/remote/smithi123/log 2024-09-16T17:36:54.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T17:36:58.770 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T17:36:58.778 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T17:36:58.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T17:36:58.848 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T17:36:58.856 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T17:36:58.856 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T17:36:58.880 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-16T17:36:58.885 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T17:36:58.885 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-16T17:36:58.885 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 -382us[ -346us] +/- 41ms 2024-09-16T17:36:58.885 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +275us[ +275us] +/- 72ms 2024-09-16T17:36:58.885 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +397us[ +433us] +/- 63ms 2024-09-16T17:36:58.885 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T17:36:58.886 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T17:36:58.895 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T17:36:58.896 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T17:36:58.946 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T17:36:59.013 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T17:36:59.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T17:36:59.041 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T17:36:59.049 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T17:36:59.080 INFO:teuthology.task.internal:Duration was 3674.454000 seconds 2024-09-16T17:36:59.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T17:36:59.130 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T17:36:59.130 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T17:36:59.170 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T17:36:59.287 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T17:36:59.287 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-09-16T17:36:59.287 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T17:36:59.347 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T17:36:59.348 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T17:36:59.414 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T17:36:59.414 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T17:36:59.556 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T17:36:59.565 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T17:36:59.565 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T17:36:59.589 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T17:36:59.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T17:36:59.653 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-09-16T17:36:59.666 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T17:36:59.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T17:36:59.725 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T17:36:59.734 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T17:36:59.735 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/matan-2024-09-16_16:23:48-crimson-rados-wip-matanb-crimson-testing-59543-distro-crimson-smithi/7907389/remote/smithi123 2024-09-16T17:36:59.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T17:36:59.798 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T17:36:59.798 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T17:36:59.848 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T17:36:59.857 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T17:36:59.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T17:36:59.889 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T17:36:59.889 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T17:36:59.903 INFO:teuthology.orchestra.run.smithi123.stdout: 265613 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 17:36 /home/ubuntu/cephtest 2024-09-16T17:36:59.904 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T17:36:59.935 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T17:37:00.004 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{clusters/fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore tasks/rbd_api_tests} duration: 3674.453999519348 flavor: crimson owner: scheduled_matan@teuthology success: true 2024-09-16T17:37:00.004 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T17:37:00.076 INFO:teuthology.run:pass