2023-01-24T10:35:53.258 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T10:35:53.259 DEBUG:teuthology.run:Teuthology command: teuthology --name bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi --verbose --owner scheduled_bcs-ceph@teuthology --description rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} --archive /home/teuthworker/archive/bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi/7134893 -- /home/teuthworker/archive/bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi/7134893/orig.config.yaml 2023-01-24T10:35:53.292 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T10:35:53.357 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi/7134893 branch: wip-kcheda3-multisite-20230112 description: rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_cls_all} email: null first_in_suite: false job_id: '7134893' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-kcheda3-multisite-20230112 ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true osd_class_default_list: '*' osd_class_load_list: '*' flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - \(PG_AVAILABILITY\) sha1: 1a4043421bb637c432ee02d001db3c6ec52c48e0 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 1a4043421bb637c432ee02d001db3c6ec52c48e0 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-kcheda3-multisite-20230112 sha1: 1a4043421bb637c432ee02d001db3c6ec52c48e0 owner: scheduled_bcs-ceph@teuthology priority: 300 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 1a4043421bb637c432ee02d001db3c6ec52c48e0 sleep_before_teardown: 0 suite: rados:basic suite_branch: wip-kcheda3-multisite-20230112 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1a4043421bb637c432ee02d001db3c6ec52c48e0/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1a4043421bb637c432ee02d001db3c6ec52c48e0 targets: smithi078.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi182.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_system_packages: deb: - sqlite3 rpm: - sqlite-devel - ceph: null - workunit: clients: client.0: - cls teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_07:57:38 tube: smithi user: bcs-ceph verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T10:35:53.358 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1a4043421bb637c432ee02d001db3c6ec52c48e0/qa; will attempt to use it 2023-01-24T10:35:53.358 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1a4043421bb637c432ee02d001db3c6ec52c48e0/qa/tasks 2023-01-24T10:35:53.359 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T10:35:53.360 INFO:teuthology.task.internal:Checking packages... 2023-01-24T10:35:53.388 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1a4043421bb637c432ee02d001db3c6ec52c48e0' 2023-01-24T10:35:53.388 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T10:35:53.389 INFO:teuthology.packaging:ref: None 2023-01-24T10:35:53.389 INFO:teuthology.packaging:tag: None 2023-01-24T10:35:53.389 INFO:teuthology.packaging:branch: wip-kcheda3-multisite-20230112 2023-01-24T10:35:53.390 INFO:teuthology.packaging:sha1: 1a4043421bb637c432ee02d001db3c6ec52c48e0 2023-01-24T10:35:53.390 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-kcheda3-multisite-20230112 2023-01-24T10:35:53.587 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1103.g1a404342 2023-01-24T10:35:53.587 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T10:35:53.593 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T10:35:53.594 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T10:35:53.622 INFO:teuthology.task.internal:Saving configuration 2023-01-24T10:35:53.634 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T10:35:53.649 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T10:35:53.670 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi/7134893', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 10:30:48.946790', 'locked_by': 'scheduled_bcs-ceph@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-24T10:35:53.693 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi182.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi/7134893', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 10:30:48.945329', 'locked_by': 'scheduled_bcs-ceph@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-24T10:35:53.693 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T10:35:53.700 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-24T10:35:53.701 INFO:teuthology.task.internal:roles: ubuntu@smithi182.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-24T10:35:53.701 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T10:35:53.780 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7cc7cf65e0>, signals=[15]) 2023-01-24T10:35:53.781 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T10:35:53.788 INFO:teuthology.task.internal:Opening connections... 2023-01-24T10:35:53.788 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2023-01-24T10:35:53.790 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T10:35:53.859 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi182.front.sepia.ceph.com 2023-01-24T10:35:53.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T10:35:53.926 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T10:35:53.934 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2023-01-24T10:35:53.955 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2023-01-24T10:35:53.956 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2023-01-24T10:35:54.015 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T10:35:54.015 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2023-01-24T10:35:54.016 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2023-01-24T10:35:54.016 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2023-01-24T10:35:54.016 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2023-01-24T10:35:54.016 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2023-01-24T10:35:54.017 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-24T10:35:54.017 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2023-01-24T10:35:54.017 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-24T10:35:54.017 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T10:35:54.017 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T10:35:54.017 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T10:35:54.018 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:35:54.018 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T10:35:54.018 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-24T10:35:54.018 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T10:35:54.018 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-24T10:35:54.019 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2023-01-24T10:35:54.039 DEBUG:teuthology.orchestra.run.smithi182:> uname -m 2023-01-24T10:35:54.059 INFO:teuthology.orchestra.run.smithi182.stdout:x86_64 2023-01-24T10:35:54.059 DEBUG:teuthology.orchestra.run.smithi182:> cat /etc/os-release 2023-01-24T10:35:54.118 INFO:teuthology.orchestra.run.smithi182.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T10:35:54.118 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION="8.6 (Ootpa)" 2023-01-24T10:35:54.118 INFO:teuthology.orchestra.run.smithi182.stdout:ID="rhel" 2023-01-24T10:35:54.119 INFO:teuthology.orchestra.run.smithi182.stdout:ID_LIKE="fedora" 2023-01-24T10:35:54.119 INFO:teuthology.orchestra.run.smithi182.stdout:VERSION_ID="8.6" 2023-01-24T10:35:54.119 INFO:teuthology.orchestra.run.smithi182.stdout:PLATFORM_ID="platform:el8" 2023-01-24T10:35:54.119 INFO:teuthology.orchestra.run.smithi182.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-24T10:35:54.119 INFO:teuthology.orchestra.run.smithi182.stdout:ANSI_COLOR="0;31" 2023-01-24T10:35:54.120 INFO:teuthology.orchestra.run.smithi182.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-24T10:35:54.120 INFO:teuthology.orchestra.run.smithi182.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T10:35:54.120 INFO:teuthology.orchestra.run.smithi182.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T10:35:54.120 INFO:teuthology.orchestra.run.smithi182.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T10:35:54.121 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:35:54.121 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T10:35:54.121 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-24T10:35:54.121 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T10:35:54.121 INFO:teuthology.orchestra.run.smithi182.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-24T10:35:54.123 INFO:teuthology.lock.ops:Updating smithi182.front.sepia.ceph.com on lock server 2023-01-24T10:35:54.141 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T10:35:54.150 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T10:35:54.172 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T10:35:54.173 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2023-01-24T10:35:54.175 DEBUG:teuthology.orchestra.run.smithi182:> test '!' -e /home/ubuntu/cephtest 2023-01-24T10:35:54.193 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T10:35:54.200 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T10:35:54.200 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2023-01-24T10:35:54.234 DEBUG:teuthology.orchestra.run.smithi182:> test -z $(ls -A /var/lib/ceph) 2023-01-24T10:35:54.257 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T10:35:54.326 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T10:35:54.341 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T10:35:54.342 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T10:35:54.342 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T10:35:54.343 DEBUG:teuthology.orchestra.run.smithi182:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T10:35:54.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T10:35:54.361 DEBUG:teuthology.orchestra.run.smithi182:> uname -r 2023-01-24T10:35:54.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T10:35:54.362 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2023-01-24T10:35:54.419 INFO:teuthology.orchestra.run.smithi182.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-24T10:35:54.419 INFO:teuthology.task.kernel:Running kernel on smithi182: 4.18.0-372.9.1.el8.x86_64 2023-01-24T10:35:54.420 DEBUG:teuthology.orchestra.run.smithi182:> sudo yum install -y kernel 2023-01-24T10:35:54.421 INFO:teuthology.orchestra.run.smithi078.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-24T10:35:54.421 INFO:teuthology.task.kernel:Running kernel on smithi078: 4.18.0-372.9.1.el8.x86_64 2023-01-24T10:35:54.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2023-01-24T10:35:55.636 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-01-24T10:35:55.636 INFO:teuthology.orchestra.run.smithi182.stdout:Unable to read consumer identity 2023-01-24T10:35:55.663 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-01-24T10:35:55.663 INFO:teuthology.orchestra.run.smithi078.stdout:Unable to read consumer identity 2023-01-24T10:35:56.973 INFO:teuthology.orchestra.run.smithi182.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-01-24T10:35:57.072 INFO:teuthology.orchestra.run.smithi078.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-01-24T10:35:58.207 INFO:teuthology.orchestra.run.smithi182.stdout:Extra Packages for Enterprise Linux 11 MB/s | 13 MB 00:01 2023-01-24T10:35:58.780 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 7.9 MB/s | 13 MB 00:01 2023-01-24T10:36:01.182 INFO:teuthology.orchestra.run.smithi182.stdout:lab-extras 445 kB/s | 24 kB 00:00 2023-01-24T10:36:01.815 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 438 kB/s | 24 kB 00:00 2023-01-24T10:36:01.826 INFO:teuthology.orchestra.run.smithi182.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-24T10:36:01.872 INFO:teuthology.orchestra.run.smithi182.stdout:Dependencies resolved. 2023-01-24T10:36:01.873 INFO:teuthology.orchestra.run.smithi182.stdout:Nothing to do. 2023-01-24T10:36:01.873 INFO:teuthology.orchestra.run.smithi182.stdout:Complete! 2023-01-24T10:36:02.007 DEBUG:teuthology.orchestra.run.smithi182:> echo no | sudo yum reinstall kernel || true 2023-01-24T10:36:02.336 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-01-24T10:36:02.336 INFO:teuthology.orchestra.run.smithi182.stdout:Unable to read consumer identity 2023-01-24T10:36:02.467 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-24T10:36:02.513 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2023-01-24T10:36:02.514 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2023-01-24T10:36:02.514 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2023-01-24T10:36:02.590 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:00:01 ago on Tue 24 Jan 2023 10:36:01 AM UTC. 2023-01-24T10:36:02.627 DEBUG:teuthology.orchestra.run.smithi078:> echo no | sudo yum reinstall kernel || true 2023-01-24T10:36:02.630 INFO:teuthology.orchestra.run.smithi182.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-24T10:36:02.676 INFO:teuthology.orchestra.run.smithi182.stderr:Error: No packages marked for reinstall. 2023-01-24T10:36:02.723 DEBUG:teuthology.orchestra.run.smithi182:> sudo yum reinstall -y kernel || true 2023-01-24T10:36:02.997 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-01-24T10:36:02.997 INFO:teuthology.orchestra.run.smithi078.stdout:Unable to read consumer identity 2023-01-24T10:36:03.050 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-01-24T10:36:03.050 INFO:teuthology.orchestra.run.smithi182.stdout:Unable to read consumer identity 2023-01-24T10:36:03.253 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 10:36:01 AM UTC. 2023-01-24T10:36:03.293 INFO:teuthology.orchestra.run.smithi078.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-24T10:36:03.299 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 10:36:01 AM UTC. 2023-01-24T10:36:03.331 INFO:teuthology.orchestra.run.smithi078.stderr:Error: No packages marked for reinstall. 2023-01-24T10:36:03.341 INFO:teuthology.orchestra.run.smithi182.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-24T10:36:03.372 INFO:teuthology.orchestra.run.smithi182.stderr:Error: No packages marked for reinstall. 2023-01-24T10:36:03.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum reinstall -y kernel || true 2023-01-24T10:36:03.418 DEBUG:teuthology.orchestra.run.smithi182:> rpm -q kernel --last | head -n 1 2023-01-24T10:36:03.497 INFO:teuthology.orchestra.run.smithi182.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-24T10:36:03.499 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-24T10:36:03.499 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T10:36:03.499 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-24T10:36:03.500 DEBUG:teuthology.orchestra.run.smithi182:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T10:36:03.554 INFO:teuthology.orchestra.run.smithi182.stdout:ttyS1 2023-01-24T10:36:03.573 DEBUG:teuthology.parallel:result is None 2023-01-24T10:36:03.702 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-01-24T10:36:03.702 INFO:teuthology.orchestra.run.smithi078.stdout:Unable to read consumer identity 2023-01-24T10:36:03.960 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 10:36:01 AM UTC. 2023-01-24T10:36:03.999 INFO:teuthology.orchestra.run.smithi078.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-24T10:36:04.021 INFO:teuthology.orchestra.run.smithi078.stderr:Error: No packages marked for reinstall. 2023-01-24T10:36:04.066 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel --last | head -n 1 2023-01-24T10:36:04.146 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-24T10:36:04.147 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-24T10:36:04.148 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T10:36:04.148 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T10:36:04.148 DEBUG:teuthology.orchestra.run.smithi078:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T10:36:04.177 INFO:teuthology.orchestra.run.smithi078.stdout:ttyS1 2023-01-24T10:36:04.190 DEBUG:teuthology.parallel:result is None 2023-01-24T10:36:04.190 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T10:36:04.197 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T10:36:04.197 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T10:36:04.232 DEBUG:teuthology.orchestra.run.smithi182:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T10:36:04.251 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T10:36:04.262 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T10:36:04.293 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T10:36:04.294 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T10:36:04.297 DEBUG:teuthology.orchestra.run.smithi182:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T10:36:04.320 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T10:36:04.331 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T10:36:04.331 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T10:36:04.354 DEBUG:teuthology.orchestra.run.smithi182:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T10:36:04.390 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T10:36:04.398 INFO:teuthology.orchestra.run.smithi182.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T10:36:04.400 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T10:36:04.408 INFO:teuthology.orchestra.run.smithi182.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T10:36:04.410 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T10:36:04.418 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T10:36:04.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T10:36:04.443 DEBUG:teuthology.orchestra.run.smithi182:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T10:36:04.478 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T10:36:04.491 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T10:36:04.491 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T10:36:04.510 DEBUG:teuthology.orchestra.run.smithi182:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T10:36:04.535 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T10:36:04.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T10:36:04.663 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T10:36:04.751 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T10:36:04.787 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:36:04.788 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T10:36:04.854 DEBUG:teuthology.orchestra.run.smithi182:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T10:36:04.897 DEBUG:teuthology.orchestra.run.smithi182:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T10:36:04.973 DEBUG:teuthology.orchestra.run.smithi182:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T10:36:05.014 DEBUG:teuthology.orchestra.run.smithi182:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T10:36:05.059 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:36:05.059 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T10:36:05.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2023-01-24T10:36:05.129 DEBUG:teuthology.orchestra.run.smithi182:> sudo service rsyslog restart 2023-01-24T10:36:05.186 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T10:36:05.228 INFO:teuthology.orchestra.run.smithi182.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T10:36:05.667 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T10:36:05.679 INFO:teuthology.task.internal:Starting timer... 2023-01-24T10:36:05.680 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T10:36:05.721 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T10:36:05.755 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T10:36:05.756 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2023-01-24T10:36:05.817 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2023-01-24T10:36:05.819 DEBUG:teuthology.orchestra.run.smithi182:> sudo service auditd rotate 2023-01-24T10:36:05.885 INFO:teuthology.orchestra.run.smithi182.stdout:Rotating logs: 2023-01-24T10:36:05.887 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T10:36:05.888 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2023-01-24T10:36:05.910 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2023-01-24T10:36:05.911 DEBUG:teuthology.orchestra.run.smithi182:> /usr/sbin/getenforce 2023-01-24T10:36:05.930 INFO:teuthology.orchestra.run.smithi182.stdout:Permissive 2023-01-24T10:36:05.930 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi182.front.sepia.ceph.com': 'permissive'} 2023-01-24T10:36:05.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T10:36:05.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T10:36:05.979 DEBUG:teuthology.orchestra.run.smithi182:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T10:36:06.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T10:36:06.006 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T10:36:06.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2023-01-24T10:36:06.046 DEBUG:teuthology.orchestra.run.smithi182:> sudo /usr/sbin/setenforce permissive 2023-01-24T10:36:06.074 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T10:36:06.087 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-24T10:36:06.092 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-24T10:36:06.207 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T10:36:06.225 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T10:36:06.227 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi182.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T10:42:48.448 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi182.front.sepia.ceph.com')] 2023-01-24T10:42:48.450 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2023-01-24T10:42:48.451 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T10:42:48.520 DEBUG:teuthology.orchestra.run.smithi078:> true 2023-01-24T10:42:48.586 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi182.front.sepia.ceph.com' 2023-01-24T10:42:48.587 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi182.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T10:42:48.654 DEBUG:teuthology.orchestra.run.smithi182:> true 2023-01-24T10:42:48.722 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T10:42:48.732 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T10:42:48.733 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T10:42:48.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T10:42:48.736 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T10:42:48.736 DEBUG:teuthology.orchestra.run.smithi182:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T10:42:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T10:42:48.780 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T10:42:48.794 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T10:42:48.808 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T10:42:48.851 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2023-01-24T10:42:48.862 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2023-01-24T10:42:48.874 INFO:teuthology.orchestra.run.smithi182.stderr:sudo: ntpd: command not found 2023-01-24T10:42:48.877 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T10:42:48.885 INFO:teuthology.orchestra.run.smithi182.stdout:506 Cannot talk to daemon 2023-01-24T10:42:48.890 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T10:42:48.899 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T10:42:48.913 INFO:teuthology.orchestra.run.smithi182.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T10:42:48.957 INFO:teuthology.orchestra.run.smithi078.stderr:bash: ntpq: command not found 2023-01-24T10:42:48.961 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T10:42:48.961 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2023-01-24T10:42:48.961 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.962 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.962 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.962 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.983 INFO:teuthology.orchestra.run.smithi182.stderr:bash: ntpq: command not found 2023-01-24T10:42:48.986 INFO:teuthology.orchestra.run.smithi182.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T10:42:48.987 INFO:teuthology.orchestra.run.smithi182.stdout:=============================================================================== 2023-01-24T10:42:48.987 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.987 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.987 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.987 INFO:teuthology.orchestra.run.smithi182.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T10:42:48.988 INFO:teuthology.run_tasks:Running task install... 2023-01-24T10:42:48.997 DEBUG:teuthology.task.install:project ceph 2023-01-24T10:42:48.997 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0'}} 2023-01-24T10:42:48.997 DEBUG:teuthology.task.install:config {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0'} 2023-01-24T10:42:48.998 INFO:teuthology.task.install:Using flavor: default 2023-01-24T10:42:49.004 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T10:42:49.004 INFO:teuthology.task.install:extra packages: [] 2023-01-24T10:42:49.004 DEBUG:teuthology.orchestra.run.smithi078:> sudo lsb_release -is 2023-01-24T10:42:49.044 INFO:teuthology.orchestra.run.smithi078.stdout:RedHatEnterprise 2023-01-24T10:42:49.046 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T10:42:49.046 DEBUG:teuthology.orchestra.run.smithi182:> sudo lsb_release -is 2023-01-24T10:42:49.047 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0', 'tag': None, 'wait_for_package': False} 2023-01-24T10:42:49.047 DEBUG:teuthology.orchestra.run.smithi078:> sudo lsb_release -is 2023-01-24T10:42:49.088 INFO:teuthology.orchestra.run.smithi182.stdout:RedHatEnterprise 2023-01-24T10:42:49.089 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T10:42:49.090 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0', 'tag': None, 'wait_for_package': False} 2023-01-24T10:42:49.090 DEBUG:teuthology.orchestra.run.smithi182:> sudo lsb_release -is 2023-01-24T10:42:49.125 INFO:teuthology.orchestra.run.smithi078.stdout:RedHatEnterprise 2023-01-24T10:42:49.126 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T10:42:49.127 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1a4043421bb637c432ee02d001db3c6ec52c48e0 2023-01-24T10:42:49.170 INFO:teuthology.orchestra.run.smithi182.stdout:RedHatEnterprise 2023-01-24T10:42:49.171 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T10:42:49.171 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1a4043421bb637c432ee02d001db3c6ec52c48e0 2023-01-24T10:42:49.376 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/ 2023-01-24T10:42:49.377 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1103.g1a404342 2023-01-24T10:42:49.398 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/ 2023-01-24T10:42:49.399 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1103.g1a404342 2023-01-24T10:42:49.522 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T10:42:49.522 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:42:49.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T10:42:49.525 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-kcheda3-multisite-20230112/1a4043421bb637c432ee02d001db3c6ec52c48e0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T10:42:49.525 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:42:49.525 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T10:42:49.551 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2023-01-24T10:42:49.552 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1a4043421bb637c432ee02d001db3c6ec52c48e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T10:42:49.553 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, sqlite-devel, sqlite-devel, sqlite-devel, sqlite-devel on remote rpm x86_64 2023-01-24T10:42:49.554 DEBUG:teuthology.orchestra.run.smithi182:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1a4043421bb637c432ee02d001db3c6ec52c48e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T10:42:49.619 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T10:42:49.622 DEBUG:teuthology.orchestra.run.smithi182:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T10:42:49.696 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T10:42:49.699 DEBUG:teuthology.orchestra.run.smithi182:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T10:42:49.760 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2023-01-24T10:42:49.762 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2023-01-24T10:42:49.764 INFO:teuthology.orchestra.run.smithi182.stdout:check_obsoletes = 1 2023-01-24T10:42:49.765 DEBUG:teuthology.orchestra.run.smithi182:> sudo yum clean all 2023-01-24T10:42:50.132 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-01-24T10:42:50.136 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-01-24T10:42:50.398 INFO:teuthology.orchestra.run.smithi182.stdout:44 files removed 2023-01-24T10:42:50.406 INFO:teuthology.orchestra.run.smithi078.stdout:44 files removed 2023-01-24T10:42:50.444 DEBUG:teuthology.orchestra.run.smithi182:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2023-01-24T10:42:50.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd sqlite-devel sqlite-devel sqlite-devel sqlite-devel 2023-01-24T10:42:50.773 INFO:teuthology.orchestra.run.smithi182.stdout:Updating Subscription Management repositories. 2023-01-24T10:42:50.781 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2023-01-24T10:42:51.410 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 309 kB/s | 81 kB 00:00 2023-01-24T10:42:51.441 INFO:teuthology.orchestra.run.smithi182.stdout:ceph packages for x86_64 292 kB/s | 81 kB 00:00 2023-01-24T10:42:51.695 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 37 kB/s | 9.3 kB 00:00 2023-01-24T10:42:51.741 INFO:teuthology.orchestra.run.smithi182.stdout:ceph noarch packages 35 kB/s | 9.3 kB 00:00 2023-01-24T10:42:51.970 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 6.0 kB/s | 1.5 kB 00:00 2023-01-24T10:42:52.016 INFO:teuthology.orchestra.run.smithi182.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2023-01-24T10:42:52.120 INFO:teuthology.orchestra.run.smithi078.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-24T10:42:52.167 INFO:teuthology.orchestra.run.smithi182.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-24T10:42:54.318 INFO:teuthology.orchestra.run.smithi182.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 13 MB 00:02 2023-01-24T10:42:54.554 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 5.5 MB/s | 13 MB 00:02 2023-01-24T10:42:57.384 INFO:teuthology.orchestra.run.smithi182.stdout:lab-extras 296 kB/s | 24 kB 00:00 2023-01-24T10:42:57.532 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 488 kB/s | 24 kB 00:00 2023-01-24T10:42:58.236 INFO:teuthology.orchestra.run.smithi182.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 60 MB/s | 47 MB 00:00 2023-01-24T10:42:58.440 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 MB/s | 47 MB 00:00 2023-01-24T10:43:06.071 INFO:teuthology.orchestra.run.smithi182.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 53 MB 00:00 2023-01-24T10:43:06.299 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 65 MB/s | 53 MB 00:00 2023-01-24T10:43:17.117 INFO:teuthology.orchestra.run.smithi182.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-01-24T10:43:17.118 INFO:teuthology.orchestra.run.smithi182.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-01-24T10:43:17.245 INFO:teuthology.orchestra.run.smithi182.stdout:Dependencies resolved. 2023-01-24T10:43:17.255 INFO:teuthology.orchestra.run.smithi182.stdout:======================================================================================================================= 2023-01-24T10:43:17.255 INFO:teuthology.orchestra.run.smithi182.stdout: Package Arch Version Repository Size 2023-01-24T10:43:17.256 INFO:teuthology.orchestra.run.smithi182.stdout:======================================================================================================================= 2023-01-24T10:43:17.256 INFO:teuthology.orchestra.run.smithi182.stdout:Installing: 2023-01-24T10:43:17.256 INFO:teuthology.orchestra.run.smithi182.stdout: ceph x86_64 2:18.0.0-1103.g1a404342.el8 ceph 6.4 k 2023-01-24T10:43:17.256 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-base x86_64 2:18.0.0-1103.g1a404342.el8 ceph 5.2 M 2023-01-24T10:43:17.256 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-fuse x86_64 2:18.0.0-1103.g1a404342.el8 ceph 845 k 2023-01-24T10:43:17.257 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1103.g1a404342.el8 ceph 165 k 2023-01-24T10:43:17.257 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr x86_64 2:18.0.0-1103.g1a404342.el8 ceph 1.5 M 2023-01-24T10:43:17.257 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 118 k 2023-01-24T10:43:17.257 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 1.7 M 2023-01-24T10:43:17.257 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 7.4 M 2023-01-24T10:43:17.258 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-rook noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 47 k 2023-01-24T10:43:17.258 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-radosgw x86_64 2:18.0.0-1103.g1a404342.el8 ceph 12 M 2023-01-24T10:43:17.258 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-test x86_64 2:18.0.0-1103.g1a404342.el8 ceph 47 M 2023-01-24T10:43:17.258 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-volume noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 263 k 2023-01-24T10:43:17.258 INFO:teuthology.orchestra.run.smithi182.stdout: cephadm noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 81 k 2023-01-24T10:43:17.259 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs-devel x86_64 2:18.0.0-1103.g1a404342.el8 ceph 30 k 2023-01-24T10:43:17.259 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs2 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 767 k 2023-01-24T10:43:17.259 INFO:teuthology.orchestra.run.smithi182.stdout: librados-devel x86_64 2:18.0.0-1103.g1a404342.el8 ceph 139 k 2023-01-24T10:43:17.259 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cephfs x86_64 2:18.0.0-1103.g1a404342.el8 ceph 190 k 2023-01-24T10:43:17.259 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rados x86_64 2:18.0.0-1103.g1a404342.el8 ceph 371 k 2023-01-24T10:43:17.259 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rbd x86_64 2:18.0.0-1103.g1a404342.el8 ceph 365 k 2023-01-24T10:43:17.260 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rgw x86_64 2:18.0.0-1103.g1a404342.el8 ceph 107 k 2023-01-24T10:43:17.260 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-fuse x86_64 2:18.0.0-1103.g1a404342.el8 ceph 94 k 2023-01-24T10:43:17.260 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-mirror x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.5 M 2023-01-24T10:43:17.260 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-nbd x86_64 2:18.0.0-1103.g1a404342.el8 ceph 179 k 2023-01-24T10:43:17.260 INFO:teuthology.orchestra.run.smithi182.stdout: sqlite-devel x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 165 k 2023-01-24T10:43:17.261 INFO:teuthology.orchestra.run.smithi182.stdout:Upgrading: 2023-01-24T10:43:17.261 INFO:teuthology.orchestra.run.smithi182.stdout: librados2 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.6 M 2023-01-24T10:43:17.261 INFO:teuthology.orchestra.run.smithi182.stdout: librbd1 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.6 M 2023-01-24T10:43:17.261 INFO:teuthology.orchestra.run.smithi182.stdout: sqlite x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 668 k 2023-01-24T10:43:17.261 INFO:teuthology.orchestra.run.smithi182.stdout: sqlite-libs x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 581 k 2023-01-24T10:43:17.262 INFO:teuthology.orchestra.run.smithi182.stdout:Installing dependencies: 2023-01-24T10:43:17.262 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-common x86_64 2:18.0.0-1103.g1a404342.el8 ceph 24 M 2023-01-24T10:43:17.262 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 23 k 2023-01-24T10:43:17.262 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mds x86_64 2:18.0.0-1103.g1a404342.el8 ceph 2.3 M 2023-01-24T10:43:17.262 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 234 k 2023-01-24T10:43:17.263 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mon x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.9 M 2023-01-24T10:43:17.263 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-osd x86_64 2:18.0.0-1103.g1a404342.el8 ceph 17 M 2023-01-24T10:43:17.263 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 14 k 2023-01-24T10:43:17.263 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-selinux x86_64 2:18.0.0-1103.g1a404342.el8 ceph 24 k 2023-01-24T10:43:17.263 INFO:teuthology.orchestra.run.smithi182.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T10:43:17.264 INFO:teuthology.orchestra.run.smithi182.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T10:43:17.264 INFO:teuthology.orchestra.run.smithi182.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-24T10:43:17.264 INFO:teuthology.orchestra.run.smithi182.stdout: libcephsqlite x86_64 2:18.0.0-1103.g1a404342.el8 ceph 137 k 2023-01-24T10:43:17.264 INFO:teuthology.orchestra.run.smithi182.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-24T10:43:17.264 INFO:teuthology.orchestra.run.smithi182.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T10:43:17.265 INFO:teuthology.orchestra.run.smithi182.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-24T10:43:17.265 INFO:teuthology.orchestra.run.smithi182.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T10:43:17.265 INFO:teuthology.orchestra.run.smithi182.stdout: libradosstriper1 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 474 k 2023-01-24T10:43:17.265 INFO:teuthology.orchestra.run.smithi182.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-24T10:43:17.265 INFO:teuthology.orchestra.run.smithi182.stdout: librgw2 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 6.4 M 2023-01-24T10:43:17.266 INFO:teuthology.orchestra.run.smithi182.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T10:43:17.266 INFO:teuthology.orchestra.run.smithi182.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T10:43:17.266 INFO:teuthology.orchestra.run.smithi182.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T10:43:17.266 INFO:teuthology.orchestra.run.smithi182.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T10:43:17.266 INFO:teuthology.orchestra.run.smithi182.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-24T10:43:17.267 INFO:teuthology.orchestra.run.smithi182.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T10:43:17.267 INFO:teuthology.orchestra.run.smithi182.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T10:43:17.267 INFO:teuthology.orchestra.run.smithi182.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T10:43:17.267 INFO:teuthology.orchestra.run.smithi182.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T10:43:17.267 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T10:43:17.268 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-argparse x86_64 2:18.0.0-1103.g1a404342.el8 ceph 45 k 2023-01-24T10:43:17.268 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-common x86_64 2:18.0.0-1103.g1a404342.el8 ceph 109 k 2023-01-24T10:43:17.268 INFO:teuthology.orchestra.run.smithi182.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T10:43:17.268 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-24T10:43:17.268 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T10:43:17.269 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T10:43:17.269 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-24T10:43:17.269 INFO:teuthology.orchestra.run.smithi182.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T10:43:17.269 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T10:43:17.269 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T10:43:17.270 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T10:43:17.270 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T10:43:17.270 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T10:43:17.270 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T10:43:17.270 INFO:teuthology.orchestra.run.smithi182.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T10:43:17.270 INFO:teuthology.orchestra.run.smithi182.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T10:43:17.271 INFO:teuthology.orchestra.run.smithi182.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T10:43:17.271 INFO:teuthology.orchestra.run.smithi182.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T10:43:17.271 INFO:teuthology.orchestra.run.smithi182.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T10:43:17.271 INFO:teuthology.orchestra.run.smithi182.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T10:43:17.271 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T10:43:17.272 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T10:43:17.272 INFO:teuthology.orchestra.run.smithi182.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T10:43:17.272 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T10:43:17.272 INFO:teuthology.orchestra.run.smithi182.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T10:43:17.272 INFO:teuthology.orchestra.run.smithi182.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T10:43:17.273 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-24T10:43:17.273 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T10:43:17.273 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T10:43:17.273 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-24T10:43:17.273 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T10:43:17.274 INFO:teuthology.orchestra.run.smithi182.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T10:43:17.274 INFO:teuthology.orchestra.run.smithi182.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T10:43:17.274 INFO:teuthology.orchestra.run.smithi182.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T10:43:17.274 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T10:43:17.274 INFO:teuthology.orchestra.run.smithi182.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-24T10:43:17.275 INFO:teuthology.orchestra.run.smithi182.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T10:43:17.275 INFO:teuthology.orchestra.run.smithi182.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T10:43:17.275 INFO:teuthology.orchestra.run.smithi182.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T10:43:17.275 INFO:teuthology.orchestra.run.smithi182.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T10:43:17.275 INFO:teuthology.orchestra.run.smithi182.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T10:43:17.276 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T10:43:17.276 INFO:teuthology.orchestra.run.smithi182.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T10:43:17.276 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T10:43:17.276 INFO:teuthology.orchestra.run.smithi182.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T10:43:17.276 INFO:teuthology.orchestra.run.smithi182.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T10:43:17.276 INFO:teuthology.orchestra.run.smithi182.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-24T10:43:17.277 INFO:teuthology.orchestra.run.smithi182.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T10:43:17.277 INFO:teuthology.orchestra.run.smithi182.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T10:43:17.277 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:43:17.277 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction Summary 2023-01-24T10:43:17.277 INFO:teuthology.orchestra.run.smithi182.stdout:======================================================================================================================= 2023-01-24T10:43:17.278 INFO:teuthology.orchestra.run.smithi182.stdout:Install 102 Packages 2023-01-24T10:43:17.278 INFO:teuthology.orchestra.run.smithi182.stdout:Upgrade 4 Packages 2023-01-24T10:43:17.278 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:43:17.279 INFO:teuthology.orchestra.run.smithi182.stdout:Total download size: 189 M 2023-01-24T10:43:17.279 INFO:teuthology.orchestra.run.smithi182.stdout:Downloading Packages: 2023-01-24T10:43:17.497 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-01-24T10:43:17.498 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-01-24T10:43:17.529 INFO:teuthology.orchestra.run.smithi182.stdout:(1/106): ceph-18.0.0-1103.g1a404342.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2023-01-24T10:43:17.628 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2023-01-24T10:43:17.638 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================================= 2023-01-24T10:43:17.638 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2023-01-24T10:43:17.638 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================================= 2023-01-24T10:43:17.638 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2023-01-24T10:43:17.638 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:18.0.0-1103.g1a404342.el8 ceph 6.4 k 2023-01-24T10:43:17.639 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:18.0.0-1103.g1a404342.el8 ceph 5.2 M 2023-01-24T10:43:17.639 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse x86_64 2:18.0.0-1103.g1a404342.el8 ceph 845 k 2023-01-24T10:43:17.639 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1103.g1a404342.el8 ceph 165 k 2023-01-24T10:43:17.639 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:18.0.0-1103.g1a404342.el8 ceph 1.5 M 2023-01-24T10:43:17.639 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 118 k 2023-01-24T10:43:17.640 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 1.7 M 2023-01-24T10:43:17.640 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 7.4 M 2023-01-24T10:43:17.640 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 47 k 2023-01-24T10:43:17.640 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:18.0.0-1103.g1a404342.el8 ceph 12 M 2023-01-24T10:43:17.641 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:18.0.0-1103.g1a404342.el8 ceph 47 M 2023-01-24T10:43:17.641 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 263 k 2023-01-24T10:43:17.641 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 81 k 2023-01-24T10:43:17.641 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:18.0.0-1103.g1a404342.el8 ceph 30 k 2023-01-24T10:43:17.641 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 767 k 2023-01-24T10:43:17.641 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:18.0.0-1103.g1a404342.el8 ceph 139 k 2023-01-24T10:43:17.642 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:18.0.0-1103.g1a404342.el8 ceph 190 k 2023-01-24T10:43:17.642 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:18.0.0-1103.g1a404342.el8 ceph 371 k 2023-01-24T10:43:17.642 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:18.0.0-1103.g1a404342.el8 ceph 365 k 2023-01-24T10:43:17.642 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:18.0.0-1103.g1a404342.el8 ceph 107 k 2023-01-24T10:43:17.642 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:18.0.0-1103.g1a404342.el8 ceph 94 k 2023-01-24T10:43:17.643 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.5 M 2023-01-24T10:43:17.643 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:18.0.0-1103.g1a404342.el8 ceph 179 k 2023-01-24T10:43:17.643 INFO:teuthology.orchestra.run.smithi078.stdout: sqlite-devel x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 165 k 2023-01-24T10:43:17.643 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2023-01-24T10:43:17.644 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.6 M 2023-01-24T10:43:17.644 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.6 M 2023-01-24T10:43:17.644 INFO:teuthology.orchestra.run.smithi078.stdout: sqlite x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 668 k 2023-01-24T10:43:17.647 INFO:teuthology.orchestra.run.smithi078.stdout: sqlite-libs x86_64 3.26.0-16.el8_6 rhel-8-for-x86_64-baseos-rpms 581 k 2023-01-24T10:43:17.648 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2023-01-24T10:43:17.648 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:18.0.0-1103.g1a404342.el8 ceph 24 M 2023-01-24T10:43:17.648 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 23 k 2023-01-24T10:43:17.648 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:18.0.0-1103.g1a404342.el8 ceph 2.3 M 2023-01-24T10:43:17.649 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 234 k 2023-01-24T10:43:17.649 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:18.0.0-1103.g1a404342.el8 ceph 3.9 M 2023-01-24T10:43:17.649 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:18.0.0-1103.g1a404342.el8 ceph 17 M 2023-01-24T10:43:17.649 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1103.g1a404342.el8 ceph-noarch 14 k 2023-01-24T10:43:17.649 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:18.0.0-1103.g1a404342.el8 ceph 24 k 2023-01-24T10:43:17.650 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T10:43:17.650 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T10:43:17.650 INFO:teuthology.orchestra.run.smithi078.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-24T10:43:17.650 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:18.0.0-1103.g1a404342.el8 ceph 137 k 2023-01-24T10:43:17.650 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-24T10:43:17.651 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T10:43:17.651 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-24T10:43:17.651 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T10:43:17.651 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 474 k 2023-01-24T10:43:17.651 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-24T10:43:17.651 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:18.0.0-1103.g1a404342.el8 ceph 6.4 M 2023-01-24T10:43:17.652 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T10:43:17.652 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T10:43:17.652 INFO:teuthology.orchestra.run.smithi078.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T10:43:17.653 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T10:43:17.653 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-24T10:43:17.653 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T10:43:17.653 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T10:43:17.653 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T10:43:17.654 INFO:teuthology.orchestra.run.smithi078.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T10:43:17.654 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T10:43:17.654 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:18.0.0-1103.g1a404342.el8 ceph 45 k 2023-01-24T10:43:17.654 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:18.0.0-1103.g1a404342.el8 ceph 109 k 2023-01-24T10:43:17.654 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T10:43:17.655 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-24T10:43:17.655 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T10:43:17.655 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T10:43:17.655 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-24T10:43:17.655 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T10:43:17.656 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T10:43:17.656 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T10:43:17.656 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T10:43:17.657 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T10:43:17.657 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T10:43:17.657 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T10:43:17.657 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T10:43:17.657 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T10:43:17.658 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T10:43:17.658 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T10:43:17.658 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T10:43:17.658 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T10:43:17.658 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T10:43:17.659 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T10:43:17.659 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T10:43:17.659 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T10:43:17.659 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T10:43:17.660 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T10:43:17.660 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-24T10:43:17.660 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T10:43:17.660 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T10:43:17.660 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-24T10:43:17.661 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T10:43:17.661 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T10:43:17.661 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T10:43:17.661 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T10:43:17.661 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T10:43:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-24T10:43:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T10:43:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T10:43:17.662 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T10:43:17.663 INFO:teuthology.orchestra.run.smithi078.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T10:43:17.663 INFO:teuthology.orchestra.run.smithi078.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T10:43:17.663 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T10:43:17.663 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T10:43:17.664 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T10:43:17.664 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T10:43:17.664 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T10:43:17.664 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-24T10:43:17.664 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T10:43:17.665 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T10:43:17.665 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:43:17.665 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2023-01-24T10:43:17.665 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================================= 2023-01-24T10:43:17.666 INFO:teuthology.orchestra.run.smithi078.stdout:Install 102 Packages 2023-01-24T10:43:17.666 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 4 Packages 2023-01-24T10:43:17.666 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:43:17.667 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 189 M 2023-01-24T10:43:17.667 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2023-01-24T10:43:17.879 INFO:teuthology.orchestra.run.smithi182.stdout:(2/106): ceph-fuse-18.0.0-1103.g1a404342.el8.x8 2.4 MB/s | 845 kB 00:00 2023-01-24T10:43:17.891 INFO:teuthology.orchestra.run.smithi078.stdout:(1/106): ceph-18.0.0-1103.g1a404342.el8.x86_64. 27 kB/s | 6.4 kB 00:00 2023-01-24T10:43:17.954 INFO:teuthology.orchestra.run.smithi182.stdout:(3/106): ceph-immutable-object-cache-18.0.0-110 2.2 MB/s | 165 kB 00:00 2023-01-24T10:43:18.291 INFO:teuthology.orchestra.run.smithi078.stdout:(2/106): ceph-fuse-18.0.0-1103.g1a404342.el8.x8 2.1 MB/s | 845 kB 00:00 2023-01-24T10:43:18.392 INFO:teuthology.orchestra.run.smithi078.stdout:(3/106): ceph-immutable-object-cache-18.0.0-110 1.6 MB/s | 165 kB 00:00 2023-01-24T10:43:18.455 INFO:teuthology.orchestra.run.smithi182.stdout:(4/106): ceph-mds-18.0.0-1103.g1a404342.el8.x86 4.5 MB/s | 2.3 MB 00:00 2023-01-24T10:43:18.522 INFO:teuthology.orchestra.run.smithi182.stdout:(5/106): ceph-base-18.0.0-1103.g1a404342.el8.x8 4.1 MB/s | 5.2 MB 00:01 2023-01-24T10:43:18.805 INFO:teuthology.orchestra.run.smithi182.stdout:(6/106): ceph-mgr-18.0.0-1103.g1a404342.el8.x86 4.4 MB/s | 1.5 MB 00:00 2023-01-24T10:43:19.000 INFO:teuthology.orchestra.run.smithi078.stdout:(4/106): ceph-mds-18.0.0-1103.g1a404342.el8.x86 3.7 MB/s | 2.3 MB 00:00 2023-01-24T10:43:19.104 INFO:teuthology.orchestra.run.smithi078.stdout:(5/106): ceph-base-18.0.0-1103.g1a404342.el8.x8 3.6 MB/s | 5.2 MB 00:01 2023-01-24T10:43:19.239 INFO:teuthology.orchestra.run.smithi182.stdout:(7/106): ceph-mon-18.0.0-1103.g1a404342.el8.x86 5.4 MB/s | 3.9 MB 00:00 2023-01-24T10:43:19.484 INFO:teuthology.orchestra.run.smithi078.stdout:(6/106): ceph-mgr-18.0.0-1103.g1a404342.el8.x86 3.2 MB/s | 1.5 MB 00:00 2023-01-24T10:43:20.034 INFO:teuthology.orchestra.run.smithi078.stdout:(7/106): ceph-mon-18.0.0-1103.g1a404342.el8.x86 4.2 MB/s | 3.9 MB 00:00 2023-01-24T10:43:21.647 INFO:teuthology.orchestra.run.smithi182.stdout:(8/106): ceph-common-18.0.0-1103.g1a404342.el8. 5.5 MB/s | 24 MB 00:04 2023-01-24T10:43:21.772 INFO:teuthology.orchestra.run.smithi182.stdout:(9/106): ceph-radosgw-18.0.0-1103.g1a404342.el8 4.9 MB/s | 12 MB 00:02 2023-01-24T10:43:21.798 INFO:teuthology.orchestra.run.smithi182.stdout:(10/106): ceph-selinux-18.0.0-1103.g1a404342.el 162 kB/s | 24 kB 00:00 2023-01-24T10:43:21.873 INFO:teuthology.orchestra.run.smithi182.stdout:(11/106): libcephfs-devel-18.0.0-1103.g1a404342 406 kB/s | 30 kB 00:00 2023-01-24T10:43:22.098 INFO:teuthology.orchestra.run.smithi182.stdout:(12/106): libcephfs2-18.0.0-1103.g1a404342.el8. 3.3 MB/s | 767 kB 00:00 2023-01-24T10:43:22.191 INFO:teuthology.orchestra.run.smithi182.stdout:(13/106): libcephsqlite-18.0.0-1103.g1a404342.e 1.5 MB/s | 137 kB 00:00 2023-01-24T10:43:22.274 INFO:teuthology.orchestra.run.smithi182.stdout:(14/106): librados-devel-18.0.0-1103.g1a404342. 1.6 MB/s | 139 kB 00:00 2023-01-24T10:43:22.443 INFO:teuthology.orchestra.run.smithi078.stdout:(8/106): ceph-radosgw-18.0.0-1103.g1a404342.el8 5.2 MB/s | 12 MB 00:02 2023-01-24T10:43:22.458 INFO:teuthology.orchestra.run.smithi182.stdout:(15/106): ceph-osd-18.0.0-1103.g1a404342.el8.x8 4.6 MB/s | 17 MB 00:03 2023-01-24T10:43:22.491 INFO:teuthology.orchestra.run.smithi182.stdout:(16/106): libradosstriper1-18.0.0-1103.g1a40434 2.1 MB/s | 474 kB 00:00 2023-01-24T10:43:22.492 INFO:teuthology.orchestra.run.smithi078.stdout:(9/106): ceph-selinux-18.0.0-1103.g1a404342.el8 490 kB/s | 24 kB 00:00 2023-01-24T10:43:22.567 INFO:teuthology.orchestra.run.smithi182.stdout:(17/106): python3-ceph-argparse-18.0.0-1103.g1a 596 kB/s | 45 kB 00:00 2023-01-24T10:43:22.650 INFO:teuthology.orchestra.run.smithi182.stdout:(18/106): python3-ceph-common-18.0.0-1103.g1a40 1.3 MB/s | 109 kB 00:00 2023-01-24T10:43:22.725 INFO:teuthology.orchestra.run.smithi182.stdout:(19/106): python3-cephfs-18.0.0-1103.g1a404342. 2.5 MB/s | 190 kB 00:00 2023-01-24T10:43:22.801 INFO:teuthology.orchestra.run.smithi182.stdout:(20/106): python3-rados-18.0.0-1103.g1a404342.e 4.8 MB/s | 371 kB 00:00 2023-01-24T10:43:22.809 INFO:teuthology.orchestra.run.smithi078.stdout:(10/106): ceph-common-18.0.0-1103.g1a404342.el8 4.7 MB/s | 24 MB 00:05 2023-01-24T10:43:22.876 INFO:teuthology.orchestra.run.smithi182.stdout:(21/106): python3-rbd-18.0.0-1103.g1a404342.el8 4.8 MB/s | 365 kB 00:00 2023-01-24T10:43:22.901 INFO:teuthology.orchestra.run.smithi078.stdout:(11/106): libcephfs-devel-18.0.0-1103.g1a404342 333 kB/s | 30 kB 00:00 2023-01-24T10:43:22.943 INFO:teuthology.orchestra.run.smithi182.stdout:(22/106): python3-rgw-18.0.0-1103.g1a404342.el8 1.6 MB/s | 107 kB 00:00 2023-01-24T10:43:23.018 INFO:teuthology.orchestra.run.smithi182.stdout:(23/106): rbd-fuse-18.0.0-1103.g1a404342.el8.x8 1.2 MB/s | 94 kB 00:00 2023-01-24T10:43:23.215 INFO:teuthology.orchestra.run.smithi078.stdout:(12/106): libcephfs2-18.0.0-1103.g1a404342.el8. 2.4 MB/s | 767 kB 00:00 2023-01-24T10:43:23.419 INFO:teuthology.orchestra.run.smithi078.stdout:(13/106): ceph-osd-18.0.0-1103.g1a404342.el8.x8 4.3 MB/s | 17 MB 00:03 2023-01-24T10:43:23.444 INFO:teuthology.orchestra.run.smithi078.stdout:(14/106): libcephsqlite-18.0.0-1103.g1a404342.e 600 kB/s | 137 kB 00:00 2023-01-24T10:43:23.503 INFO:teuthology.orchestra.run.smithi078.stdout:(15/106): librados-devel-18.0.0-1103.g1a404342. 1.6 MB/s | 139 kB 00:00 2023-01-24T10:43:23.586 INFO:teuthology.orchestra.run.smithi078.stdout:(16/106): libradosstriper1-18.0.0-1103.g1a40434 3.3 MB/s | 474 kB 00:00 2023-01-24T10:43:23.618 INFO:teuthology.orchestra.run.smithi182.stdout:(24/106): librgw2-18.0.0-1103.g1a404342.el8.x86 5.5 MB/s | 6.4 MB 00:01 2023-01-24T10:43:23.645 INFO:teuthology.orchestra.run.smithi078.stdout:(17/106): python3-ceph-argparse-18.0.0-1103.g1a 766 kB/s | 45 kB 00:00 2023-01-24T10:43:23.669 INFO:teuthology.orchestra.run.smithi182.stdout:(25/106): rbd-mirror-18.0.0-1103.g1a404342.el8. 5.4 MB/s | 3.5 MB 00:00 2023-01-24T10:43:23.696 INFO:teuthology.orchestra.run.smithi182.stdout:(26/106): rbd-nbd-18.0.0-1103.g1a404342.el8.x86 2.3 MB/s | 179 kB 00:00 2023-01-24T10:43:23.704 INFO:teuthology.orchestra.run.smithi078.stdout:(18/106): python3-ceph-common-18.0.0-1103.g1a40 1.8 MB/s | 109 kB 00:00 2023-01-24T10:43:23.728 INFO:teuthology.orchestra.run.smithi182.stdout:(27/106): ceph-grafana-dashboards-18.0.0-1103.g 391 kB/s | 23 kB 00:00 2023-01-24T10:43:23.753 INFO:teuthology.orchestra.run.smithi182.stdout:(28/106): ceph-mgr-cephadm-18.0.0-1103.g1a40434 2.0 MB/s | 118 kB 00:00 2023-01-24T10:43:23.795 INFO:teuthology.orchestra.run.smithi078.stdout:(19/106): python3-cephfs-18.0.0-1103.g1a404342. 2.0 MB/s | 190 kB 00:00 2023-01-24T10:43:23.896 INFO:teuthology.orchestra.run.smithi078.stdout:(20/106): python3-rados-18.0.0-1103.g1a404342.e 3.6 MB/s | 371 kB 00:00 2023-01-24T10:43:23.996 INFO:teuthology.orchestra.run.smithi078.stdout:(21/106): python3-rbd-18.0.0-1103.g1a404342.el8 3.6 MB/s | 365 kB 00:00 2023-01-24T10:43:24.078 INFO:teuthology.orchestra.run.smithi182.stdout:(29/106): ceph-mgr-dashboard-18.0.0-1103.g1a404 4.7 MB/s | 1.7 MB 00:00 2023-01-24T10:43:24.088 INFO:teuthology.orchestra.run.smithi078.stdout:(22/106): python3-rgw-18.0.0-1103.g1a404342.el8 1.1 MB/s | 107 kB 00:00 2023-01-24T10:43:24.154 INFO:teuthology.orchestra.run.smithi182.stdout:(30/106): ceph-mgr-modules-core-18.0.0-1103.g1a 3.0 MB/s | 234 kB 00:00 2023-01-24T10:43:24.171 INFO:teuthology.orchestra.run.smithi078.stdout:(23/106): rbd-fuse-18.0.0-1103.g1a404342.el8.x8 1.1 MB/s | 94 kB 00:00 2023-01-24T10:43:24.212 INFO:teuthology.orchestra.run.smithi182.stdout:(31/106): ceph-mgr-rook-18.0.0-1103.g1a404342.e 804 kB/s | 47 kB 00:00 2023-01-24T10:43:24.296 INFO:teuthology.orchestra.run.smithi182.stdout:(32/106): ceph-prometheus-alerts-18.0.0-1103.g1 174 kB/s | 14 kB 00:00 2023-01-24T10:43:24.388 INFO:teuthology.orchestra.run.smithi182.stdout:(33/106): ceph-volume-18.0.0-1103.g1a404342.el8 2.8 MB/s | 263 kB 00:00 2023-01-24T10:43:24.472 INFO:teuthology.orchestra.run.smithi182.stdout:(34/106): cephadm-18.0.0-1103.g1a404342.el8.noa 976 kB/s | 81 kB 00:00 2023-01-24T10:43:24.707 INFO:teuthology.orchestra.run.smithi182.stdout:(35/106): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-01-24T10:43:24.755 INFO:teuthology.orchestra.run.smithi182.stdout:(36/106): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2023-01-24T10:43:24.780 INFO:teuthology.orchestra.run.smithi078.stdout:(24/106): librgw2-18.0.0-1103.g1a404342.el8.x86 5.0 MB/s | 6.4 MB 00:01 2023-01-24T10:43:24.797 INFO:teuthology.orchestra.run.smithi182.stdout:(37/106): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-01-24T10:43:24.847 INFO:teuthology.orchestra.run.smithi078.stdout:(25/106): rbd-mirror-18.0.0-1103.g1a404342.el8. 5.2 MB/s | 3.5 MB 00:00 2023-01-24T10:43:24.872 INFO:teuthology.orchestra.run.smithi078.stdout:(26/106): rbd-nbd-18.0.0-1103.g1a404342.el8.x86 1.9 MB/s | 179 kB 00:00 2023-01-24T10:43:24.881 INFO:teuthology.orchestra.run.smithi182.stdout:(38/106): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2023-01-24T10:43:24.931 INFO:teuthology.orchestra.run.smithi182.stdout:(39/106): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 874 kB/s | 44 kB 00:00 2023-01-24T10:43:24.932 INFO:teuthology.orchestra.run.smithi078.stdout:(27/106): ceph-grafana-dashboards-18.0.0-1103.g 270 kB/s | 23 kB 00:00 2023-01-24T10:43:24.965 INFO:teuthology.orchestra.run.smithi078.stdout:(28/106): ceph-mgr-cephadm-18.0.0-1103.g1a40434 1.3 MB/s | 118 kB 00:00 2023-01-24T10:43:24.973 INFO:teuthology.orchestra.run.smithi182.stdout:(40/106): python3-beautifulsoup4-4.6.3-2.el8.1. 4.4 MB/s | 185 kB 00:00 2023-01-24T10:43:25.024 INFO:teuthology.orchestra.run.smithi182.stdout:(41/106): python3-cachetools-3.1.1-4.el8.noarch 664 kB/s | 33 kB 00:00 2023-01-24T10:43:25.074 INFO:teuthology.orchestra.run.smithi182.stdout:(42/106): python3-certifi-2018.10.15-7.el8.noar 323 kB/s | 16 kB 00:00 2023-01-24T10:43:25.166 INFO:teuthology.orchestra.run.smithi182.stdout:(43/106): ceph-mgr-diskprediction-local-18.0.0- 5.2 MB/s | 7.4 MB 00:01 2023-01-24T10:43:25.191 INFO:teuthology.orchestra.run.smithi182.stdout:(44/106): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-01-24T10:43:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:(29/106): ceph-mgr-dashboard-18.0.0-1103.g1a404 5.7 MB/s | 1.7 MB 00:00 2023-01-24T10:43:25.233 INFO:teuthology.orchestra.run.smithi182.stdout:(45/106): python3-google-auth-1.1.1-10.el8.noar 1.9 MB/s | 81 kB 00:00 2023-01-24T10:43:25.284 INFO:teuthology.orchestra.run.smithi182.stdout:(46/106): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-01-24T10:43:25.290 INFO:teuthology.orchestra.run.smithi078.stdout:(30/106): ceph-mgr-modules-core-18.0.0-1103.g1a 3.4 MB/s | 234 kB 00:00 2023-01-24T10:43:25.334 INFO:teuthology.orchestra.run.smithi182.stdout:(47/106): python3-jaraco-functools-2.0-4.el8.no 352 kB/s | 18 kB 00:00 2023-01-24T10:43:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:(31/106): ceph-mgr-rook-18.0.0-1103.g1a404342.e 705 kB/s | 47 kB 00:00 2023-01-24T10:43:25.376 INFO:teuthology.orchestra.run.smithi182.stdout:(48/106): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-01-24T10:43:25.424 INFO:teuthology.orchestra.run.smithi078.stdout:(32/106): ceph-prometheus-alerts-18.0.0-1103.g1 218 kB/s | 14 kB 00:00 2023-01-24T10:43:25.426 INFO:teuthology.orchestra.run.smithi182.stdout:(49/106): python3-logutils-0.3.5-11.el8.noarch. 972 kB/s | 49 kB 00:00 2023-01-24T10:43:25.477 INFO:teuthology.orchestra.run.smithi182.stdout:(50/106): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2023-01-24T10:43:25.525 INFO:teuthology.orchestra.run.smithi078.stdout:(33/106): ceph-volume-18.0.0-1103.g1a404342.el8 2.6 MB/s | 263 kB 00:00 2023-01-24T10:43:25.535 INFO:teuthology.orchestra.run.smithi182.stdout:(51/106): python3-kubernetes-11.0.0-6.el8.noarc 5.8 MB/s | 1.2 MB 00:00 2023-01-24T10:43:25.561 INFO:teuthology.orchestra.run.smithi182.stdout:(52/106): python3-natsort-7.1.1-2.el8.noarch.rp 722 kB/s | 60 kB 00:00 2023-01-24T10:43:25.574 INFO:teuthology.orchestra.run.smithi078.stdout:(34/106): cephadm-18.0.0-1103.g1a404342.el8.noa 1.6 MB/s | 81 kB 00:00 2023-01-24T10:43:25.594 INFO:teuthology.orchestra.run.smithi182.stdout:(53/106): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-01-24T10:43:25.620 INFO:teuthology.orchestra.run.smithi182.stdout:(54/106): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-01-24T10:43:25.645 INFO:teuthology.orchestra.run.smithi182.stdout:(55/106): python3-repoze-lru-0.7-6.el8.noarch.r 669 kB/s | 34 kB 00:00 2023-01-24T10:43:25.670 INFO:teuthology.orchestra.run.smithi182.stdout:(56/106): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-01-24T10:43:25.704 INFO:teuthology.orchestra.run.smithi182.stdout:(57/106): python3-rsa-4.8-1.el8.noarch.rpm 1.0 MB/s | 61 kB 00:00 2023-01-24T10:43:25.749 INFO:teuthology.orchestra.run.smithi182.stdout:(58/106): python3-simplegeneric-0.8.1-17.el8.no 241 kB/s | 19 kB 00:00 2023-01-24T10:43:25.779 INFO:teuthology.orchestra.run.smithi182.stdout:(59/106): python3-singledispatch-3.4.0.3-18.el8 319 kB/s | 24 kB 00:00 2023-01-24T10:43:25.783 INFO:teuthology.orchestra.run.smithi078.stdout:(35/106): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-01-24T10:43:25.805 INFO:teuthology.orchestra.run.smithi182.stdout:(60/106): python3-tempora-1.14.1-5.el8.noarch.r 525 kB/s | 29 kB 00:00 2023-01-24T10:43:25.831 INFO:teuthology.orchestra.run.smithi182.stdout:(61/106): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-01-24T10:43:25.833 INFO:teuthology.orchestra.run.smithi078.stdout:(36/106): liboath-2.6.2-3.el8.x86_64.rpm 1.2 MB/s | 59 kB 00:00 2023-01-24T10:43:25.855 INFO:teuthology.orchestra.run.smithi182.stdout:(62/106): python3-waitress-1.2.1-2.el8.1.noarch 4.7 MB/s | 241 kB 00:00 2023-01-24T10:43:25.876 INFO:teuthology.orchestra.run.smithi078.stdout:(37/106): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2023-01-24T10:43:25.880 INFO:teuthology.orchestra.run.smithi182.stdout:(63/106): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-01-24T10:43:25.906 INFO:teuthology.orchestra.run.smithi182.stdout:(64/106): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-01-24T10:43:25.923 INFO:teuthology.orchestra.run.smithi182.stdout:(65/106): python3-webtest-2.0.33-1.el8.noarch.r 2.0 MB/s | 85 kB 00:00 2023-01-24T10:43:25.957 INFO:teuthology.orchestra.run.smithi182.stdout:(66/106): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2023-01-24T10:43:25.982 INFO:teuthology.orchestra.run.smithi182.stdout:(67/106): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 70 kB 00:00 2023-01-24T10:43:26.018 INFO:teuthology.orchestra.run.smithi078.stdout:(38/106): ceph-mgr-diskprediction-local-18.0.0- 7.0 MB/s | 7.4 MB 00:01 2023-01-24T10:43:26.043 INFO:teuthology.orchestra.run.smithi078.stdout:(39/106): python3-asyncssh-2.7.0-2.el8.noarch.r 2.8 MB/s | 476 kB 00:00 2023-01-24T10:43:26.101 INFO:teuthology.orchestra.run.smithi078.stdout:(40/106): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-01-24T10:43:26.132 INFO:teuthology.orchestra.run.smithi182.stdout:(68/106): thrift-0.13.0-2.el8.x86_64.rpm 8.3 MB/s | 1.7 MB 00:00 2023-01-24T10:43:26.143 INFO:teuthology.orchestra.run.smithi078.stdout:(41/106): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 348 kB/s | 44 kB 00:00 2023-01-24T10:43:26.160 INFO:teuthology.orchestra.run.smithi078.stdout:(42/106): python3-cachetools-3.1.1-4.el8.noarch 565 kB/s | 33 kB 00:00 2023-01-24T10:43:26.185 INFO:teuthology.orchestra.run.smithi078.stdout:(43/106): python3-certifi-2018.10.15-7.el8.noar 390 kB/s | 16 kB 00:00 2023-01-24T10:43:26.208 INFO:teuthology.orchestra.run.smithi182.stdout:(69/106): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-01-24T10:43:26.219 INFO:teuthology.orchestra.run.smithi078.stdout:(44/106): python3-cheroot-8.5.2-1.el8.noarch.rp 2.9 MB/s | 173 kB 00:00 2023-01-24T10:43:26.241 INFO:teuthology.orchestra.run.smithi182.stdout:(70/106): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-01-24T10:43:26.269 INFO:teuthology.orchestra.run.smithi078.stdout:(45/106): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-01-24T10:43:26.320 INFO:teuthology.orchestra.run.smithi078.stdout:(46/106): python3-cherrypy-18.4.0-1.el8.noarch. 2.8 MB/s | 384 kB 00:00 2023-01-24T10:43:26.345 INFO:teuthology.orchestra.run.smithi078.stdout:(47/106): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-01-24T10:43:26.387 INFO:teuthology.orchestra.run.smithi078.stdout:(48/106): python3-jaraco-functools-2.0-4.el8.no 263 kB/s | 18 kB 00:00 2023-01-24T10:43:26.400 INFO:teuthology.orchestra.run.smithi182.stdout:(71/106): python3-scipy-1.0.0-21.module+el8.5.0 33 MB/s | 14 MB 00:00 2023-01-24T10:43:26.425 INFO:teuthology.orchestra.run.smithi182.stdout:(72/106): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2023-01-24T10:43:26.446 INFO:teuthology.orchestra.run.smithi078.stdout:(49/106): python3-logutils-0.3.5-11.el8.noarch. 837 kB/s | 49 kB 00:00 2023-01-24T10:43:26.459 INFO:teuthology.orchestra.run.smithi182.stdout:(73/106): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-01-24T10:43:26.484 INFO:teuthology.orchestra.run.smithi182.stdout:(74/106): python3-markupsafe-0.23-19.el8.x86_64 664 kB/s | 39 kB 00:00 2023-01-24T10:43:26.496 INFO:teuthology.orchestra.run.smithi078.stdout:(50/106): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2023-01-24T10:43:26.518 INFO:teuthology.orchestra.run.smithi182.stdout:(75/106): python3-jinja2-2.10.1-3.el8.noarch.rp 9.0 MB/s | 538 kB 00:00 2023-01-24T10:43:26.538 INFO:teuthology.orchestra.run.smithi078.stdout:(51/106): python3-kubernetes-11.0.0-6.el8.noarc 6.1 MB/s | 1.2 MB 00:00 2023-01-24T10:43:26.545 INFO:teuthology.orchestra.run.smithi182.stdout:(76/106): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-24T10:43:26.563 INFO:teuthology.orchestra.run.smithi078.stdout:(52/106): python3-natsort-7.1.1-2.el8.noarch.rp 902 kB/s | 60 kB 00:00 2023-01-24T10:43:26.568 INFO:teuthology.orchestra.run.smithi182.stdout:(77/106): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-01-24T10:43:26.597 INFO:teuthology.orchestra.run.smithi078.stdout:(53/106): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-01-24T10:43:26.610 INFO:teuthology.orchestra.run.smithi182.stdout:(78/106): python3-jsonpatch-1.21-2.el8.noarch.r 637 kB/s | 27 kB 00:00 2023-01-24T10:43:26.622 INFO:teuthology.orchestra.run.smithi078.stdout:(54/106): python3-portend-2.6-1.el8.noarch.rpm 266 kB/s | 16 kB 00:00 2023-01-24T10:43:26.661 INFO:teuthology.orchestra.run.smithi182.stdout:(79/106): python3-prettytable-0.7.2-14.el8.noar 883 kB/s | 44 kB 00:00 2023-01-24T10:43:26.672 INFO:teuthology.orchestra.run.smithi078.stdout:(55/106): python3-repoze-lru-0.7-6.el8.noarch.r 446 kB/s | 34 kB 00:00 2023-01-24T10:43:26.698 INFO:teuthology.orchestra.run.smithi078.stdout:(56/106): python3-routes-2.4.1-12.el8.noarch.rp 2.5 MB/s | 196 kB 00:00 2023-01-24T10:43:26.723 INFO:teuthology.orchestra.run.smithi078.stdout:(57/106): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2023-01-24T10:43:26.736 INFO:teuthology.orchestra.run.smithi182.stdout:(80/106): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-01-24T10:43:26.748 INFO:teuthology.orchestra.run.smithi078.stdout:(58/106): python3-simplegeneric-0.8.1-17.el8.no 373 kB/s | 19 kB 00:00 2023-01-24T10:43:26.773 INFO:teuthology.orchestra.run.smithi078.stdout:(59/106): python3-singledispatch-3.4.0.3-18.el8 477 kB/s | 24 kB 00:00 2023-01-24T10:43:26.778 INFO:teuthology.orchestra.run.smithi182.stdout:(81/106): python3-werkzeug-0.12.2-4.el8.noarch. 11 MB/s | 457 kB 00:00 2023-01-24T10:43:26.799 INFO:teuthology.orchestra.run.smithi078.stdout:(60/106): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2023-01-24T10:43:26.825 INFO:teuthology.orchestra.run.smithi078.stdout:(61/106): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-01-24T10:43:26.853 INFO:teuthology.orchestra.run.smithi182.stdout:(82/106): python3-numpy-1.14.3-10.el8.x86_64.rp 19 MB/s | 3.7 MB 00:00 2023-01-24T10:43:26.858 INFO:teuthology.orchestra.run.smithi078.stdout:(62/106): python3-waitress-1.2.1-2.el8.1.noarch 4.0 MB/s | 241 kB 00:00 2023-01-24T10:43:26.878 INFO:teuthology.orchestra.run.smithi182.stdout:(83/106): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.2 MB/s | 225 kB 00:00 2023-01-24T10:43:26.883 INFO:teuthology.orchestra.run.smithi078.stdout:(63/106): python3-webob-1.8.5-1.el8.1.noarch.rp 4.2 MB/s | 251 kB 00:00 2023-01-24T10:43:26.904 INFO:teuthology.orchestra.run.smithi182.stdout:(84/106): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-24T10:43:26.908 INFO:teuthology.orchestra.run.smithi078.stdout:(64/106): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-01-24T10:43:26.929 INFO:teuthology.orchestra.run.smithi182.stdout:(85/106): python3-jsonpointer-1.10-11.el8.noarc 397 kB/s | 20 kB 00:00 2023-01-24T10:43:26.933 INFO:teuthology.orchestra.run.smithi078.stdout:(65/106): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-01-24T10:43:26.946 INFO:teuthology.orchestra.run.smithi182.stdout:(86/106): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2023-01-24T10:43:26.959 INFO:teuthology.orchestra.run.smithi078.stdout:(66/106): python3-zc-lockfile-2.0-2.el8.noarch. 457 kB/s | 23 kB 00:00 2023-01-24T10:43:26.984 INFO:teuthology.orchestra.run.smithi078.stdout:(67/106): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-01-24T10:43:27.089 INFO:teuthology.orchestra.run.smithi182.stdout:(87/106): openblas-threads-0.3.15-3.el8.x86_64. 31 MB/s | 4.9 MB 00:00 2023-01-24T10:43:27.160 INFO:teuthology.orchestra.run.smithi078.stdout:(68/106): thrift-0.13.0-2.el8.x86_64.rpm 7.7 MB/s | 1.7 MB 00:00 2023-01-24T10:43:27.163 INFO:teuthology.orchestra.run.smithi182.stdout:(88/106): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-01-24T10:43:27.189 INFO:teuthology.orchestra.run.smithi182.stdout:(89/106): jq-1.6-3.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2023-01-24T10:43:27.214 INFO:teuthology.orchestra.run.smithi182.stdout:(90/106): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-01-24T10:43:27.222 INFO:teuthology.orchestra.run.smithi078.stdout:(69/106): python3-pyasn1-modules-0.3.7-6.el8.no 1.7 MB/s | 110 kB 00:00 2023-01-24T10:43:27.239 INFO:teuthology.orchestra.run.smithi182.stdout:(91/106): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2023-01-24T10:43:27.264 INFO:teuthology.orchestra.run.smithi182.stdout:(92/106): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-01-24T10:43:27.290 INFO:teuthology.orchestra.run.smithi182.stdout:(93/106): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-01-24T10:43:27.315 INFO:teuthology.orchestra.run.smithi182.stdout:(94/106): python3-requests-oauthlib-1.0.0-1.el8 847 kB/s | 43 kB 00:00 2023-01-24T10:43:27.340 INFO:teuthology.orchestra.run.smithi182.stdout:(95/106): python3-jwt-1.6.1-2.el8.noarch.rpm 846 kB/s | 43 kB 00:00 2023-01-24T10:43:27.375 INFO:teuthology.orchestra.run.smithi182.stdout:(96/106): librabbitmq-0.9.0-3.el8.x86_64.rpm 793 kB/s | 47 kB 00:00 2023-01-24T10:43:27.385 INFO:teuthology.orchestra.run.smithi078.stdout:(70/106): python3-scipy-1.0.0-21.module+el8.5.0 34 MB/s | 14 MB 00:00 2023-01-24T10:43:27.408 INFO:teuthology.orchestra.run.smithi182.stdout:(97/106): python3-cryptography-3.2.1-5.el8.x86_ 8.1 MB/s | 559 kB 00:00 2023-01-24T10:43:27.410 INFO:teuthology.orchestra.run.smithi078.stdout:(71/106): python3-pytz-2017.2-9.el8.noarch.rpm 288 kB/s | 54 kB 00:00 2023-01-24T10:43:27.433 INFO:teuthology.orchestra.run.smithi182.stdout:(98/106): fuse-2.9.7-15.el8.x86_64.rpm 1.4 MB/s | 83 kB 00:00 2023-01-24T10:43:27.436 INFO:teuthology.orchestra.run.smithi078.stdout:(72/106): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-01-24T10:43:27.470 INFO:teuthology.orchestra.run.smithi078.stdout:(73/106): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-01-24T10:43:27.475 INFO:teuthology.orchestra.run.smithi182.stdout:(99/106): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-01-24T10:43:27.497 INFO:teuthology.orchestra.run.smithi078.stdout:(74/106): python3-markupsafe-0.23-19.el8.x86_64 665 kB/s | 39 kB 00:00 2023-01-24T10:43:27.500 INFO:teuthology.orchestra.run.smithi182.stdout:(100/106): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-01-24T10:43:27.536 INFO:teuthology.orchestra.run.smithi078.stdout:(75/106): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2023-01-24T10:43:27.551 INFO:teuthology.orchestra.run.smithi182.stdout:(101/106): sqlite-devel-3.26.0-16.el8_6.x86_64. 2.1 MB/s | 165 kB 00:00 2023-01-24T10:43:27.562 INFO:teuthology.orchestra.run.smithi078.stdout:(76/106): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-01-24T10:43:27.587 INFO:teuthology.orchestra.run.smithi078.stdout:(77/106): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-01-24T10:43:27.621 INFO:teuthology.orchestra.run.smithi078.stdout:(78/106): python3-jsonpatch-1.21-2.el8.noarch.r 799 kB/s | 27 kB 00:00 2023-01-24T10:43:27.654 INFO:teuthology.orchestra.run.smithi078.stdout:(79/106): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-01-24T10:43:27.754 INFO:teuthology.orchestra.run.smithi078.stdout:(80/106): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-01-24T10:43:28.010 INFO:teuthology.orchestra.run.smithi182.stdout:(102/106): ceph-test-18.0.0-1103.g1a404342.el8. 7.6 MB/s | 47 MB 00:06 2023-01-24T10:43:28.060 INFO:teuthology.orchestra.run.smithi182.stdout:(103/106): sqlite-3.26.0-16.el8_6.x86_64.rpm 13 MB/s | 668 kB 00:00 2023-01-24T10:43:28.102 INFO:teuthology.orchestra.run.smithi182.stdout:(104/106): sqlite-libs-3.26.0-16.el8_6.x86_64.r 14 MB/s | 581 kB 00:00 2023-01-24T10:43:28.235 INFO:teuthology.orchestra.run.smithi182.stdout:(105/106): librbd1-18.0.0-1103.g1a404342.el8.x8 5.3 MB/s | 3.6 MB 00:00 2023-01-24T10:43:28.255 INFO:teuthology.orchestra.run.smithi078.stdout:(81/106): ceph-test-18.0.0-1103.g1a404342.el8.x 8.2 MB/s | 47 MB 00:05 2023-01-24T10:43:28.294 INFO:teuthology.orchestra.run.smithi182.stdout:(106/106): librados2-18.0.0-1103.g1a404342.el8. 4.6 MB/s | 3.6 MB 00:00 2023-01-24T10:43:28.297 INFO:teuthology.orchestra.run.smithi182.stdout:-------------------------------------------------------------------------------- 2023-01-24T10:43:28.297 INFO:teuthology.orchestra.run.smithi182.stdout:Total 17 MB/s | 189 MB 00:11 2023-01-24T10:43:28.419 INFO:teuthology.orchestra.run.smithi078.stdout:(82/106): python3-werkzeug-0.12.2-4.el8.noarch. 689 kB/s | 457 kB 00:00 2023-01-24T10:43:28.499 INFO:teuthology.orchestra.run.smithi078.stdout:(83/106): python3-numpy-1.14.3-10.el8.x86_64.rp 4.4 MB/s | 3.7 MB 00:00 2023-01-24T10:43:28.524 INFO:teuthology.orchestra.run.smithi078.stdout:(84/106): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2023-01-24T10:43:28.550 INFO:teuthology.orchestra.run.smithi078.stdout:(85/106): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-01-24T10:43:28.575 INFO:teuthology.orchestra.run.smithi078.stdout:(86/106): python3-numpy-f2py-1.14.3-10.el8.x86_ 704 kB/s | 225 kB 00:00 2023-01-24T10:43:28.601 INFO:teuthology.orchestra.run.smithi078.stdout:(87/106): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-01-24T10:43:28.642 INFO:teuthology.orchestra.run.smithi078.stdout:(88/106): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-01-24T10:43:28.684 INFO:teuthology.orchestra.run.smithi078.stdout:(89/106): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-01-24T10:43:28.726 INFO:teuthology.orchestra.run.smithi078.stdout:(90/106): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-01-24T10:43:28.793 INFO:teuthology.orchestra.run.smithi078.stdout:(91/106): openblas-0.3.15-3.el8.x86_64.rpm 22 MB/s | 4.7 MB 00:00 2023-01-24T10:43:28.839 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction check 2023-01-24T10:43:28.860 INFO:teuthology.orchestra.run.smithi078.stdout:(92/106): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-01-24T10:43:28.885 INFO:teuthology.orchestra.run.smithi078.stdout:(93/106): python3-pycparser-2.14-14.el8.noarch. 686 kB/s | 109 kB 00:00 2023-01-24T10:43:28.912 INFO:teuthology.orchestra.run.smithi078.stdout:(94/106): python3-requests-oauthlib-1.0.0-1.el8 853 kB/s | 43 kB 00:00 2023-01-24T10:43:28.937 INFO:teuthology.orchestra.run.smithi078.stdout:(95/106): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-01-24T10:43:28.939 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction check succeeded. 2023-01-24T10:43:28.940 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction test 2023-01-24T10:43:28.961 INFO:teuthology.orchestra.run.smithi078.stdout:(96/106): python3-oauthlib-2.1.0-1.el8.noarch.r 923 kB/s | 155 kB 00:00 2023-01-24T10:43:28.987 INFO:teuthology.orchestra.run.smithi078.stdout:(97/106): librabbitmq-0.9.0-3.el8.x86_64.rpm 619 kB/s | 47 kB 00:00 2023-01-24T10:43:29.029 INFO:teuthology.orchestra.run.smithi078.stdout:(98/106): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-01-24T10:43:29.062 INFO:teuthology.orchestra.run.smithi078.stdout:(99/106): python3-cryptography-3.2.1-5.el8.x86_ 4.3 MB/s | 559 kB 00:00 2023-01-24T10:43:29.096 INFO:teuthology.orchestra.run.smithi078.stdout:(100/106): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-01-24T10:43:29.129 INFO:teuthology.orchestra.run.smithi078.stdout:(101/106): libgfortran-8.5.0-10.1.el8_6.x86_64. 4.4 MB/s | 644 kB 00:00 2023-01-24T10:43:29.155 INFO:teuthology.orchestra.run.smithi078.stdout:(102/106): sqlite-devel-3.26.0-16.el8_6.x86_64. 1.7 MB/s | 165 kB 00:00 2023-01-24T10:43:29.197 INFO:teuthology.orchestra.run.smithi078.stdout:(103/106): sqlite-3.26.0-16.el8_6.x86_64.rpm 16 MB/s | 668 kB 00:00 2023-01-24T10:43:29.247 INFO:teuthology.orchestra.run.smithi078.stdout:(104/106): sqlite-libs-3.26.0-16.el8_6.x86_64.r 11 MB/s | 581 kB 00:00 2023-01-24T10:43:29.522 INFO:teuthology.orchestra.run.smithi078.stdout:(105/106): librados2-18.0.0-1103.g1a404342.el8. 8.5 MB/s | 3.6 MB 00:00 2023-01-24T10:43:29.600 INFO:teuthology.orchestra.run.smithi078.stdout:(106/106): librbd1-18.0.0-1103.g1a404342.el8.x8 7.7 MB/s | 3.6 MB 00:00 2023-01-24T10:43:29.602 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2023-01-24T10:43:29.603 INFO:teuthology.orchestra.run.smithi078.stdout:Total 16 MB/s | 189 MB 00:11 2023-01-24T10:43:29.911 INFO:teuthology.orchestra.run.smithi182.stdout:Transaction test succeeded. 2023-01-24T10:43:29.915 INFO:teuthology.orchestra.run.smithi182.stdout:Running transaction 2023-01-24T10:43:30.145 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2023-01-24T10:43:30.246 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2023-01-24T10:43:30.247 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2023-01-24T10:43:31.205 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2023-01-24T10:43:31.208 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2023-01-24T10:43:33.916 INFO:teuthology.orchestra.run.smithi182.stdout: Preparing : 1/1 2023-01-24T10:43:34.141 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/110 2023-01-24T10:43:34.347 INFO:teuthology.orchestra.run.smithi182.stdout: Upgrading : sqlite-libs-3.26.0-16.el8_6.x86_64 2/110 2023-01-24T10:43:34.520 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/110 2023-01-24T10:43:34.739 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/110 2023-01-24T10:43:35.133 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/110 2023-01-24T10:43:35.250 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/110 2023-01-24T10:43:35.450 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/110 2023-01-24T10:43:35.583 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2023-01-24T10:43:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/110 2023-01-24T10:43:35.846 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/110 2023-01-24T10:43:36.006 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : sqlite-libs-3.26.0-16.el8_6.x86_64 2/110 2023-01-24T10:43:36.083 INFO:teuthology.orchestra.run.smithi182.stdout: Upgrading : librados2-2:18.0.0-1103.g1a404342.el8.x86_64 7/110 2023-01-24T10:43:36.161 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/110 2023-01-24T10:43:36.358 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/110 2023-01-24T10:43:36.489 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librados2-2:18.0.0-1103.g1a404342.el8.x86_64 7/110 2023-01-24T10:43:36.551 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/110 2023-01-24T10:43:36.674 INFO:teuthology.orchestra.run.smithi182.stdout: Upgrading : librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 8/110 2023-01-24T10:43:36.676 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/110 2023-01-24T10:43:36.803 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 8/110 2023-01-24T10:43:36.867 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/110 2023-01-24T10:43:37.008 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 9/110 2023-01-24T10:43:37.092 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 9/110 2023-01-24T10:43:37.264 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/110 2023-01-24T10:43:37.318 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rados-2:18.0.0-1103.g1a404342.el8.x86_64 10/110 2023-01-24T10:43:37.484 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:18.0.0-1103.g1a404342.el8.x86_64 7/110 2023-01-24T10:43:37.518 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/110 2023-01-24T10:43:37.702 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/110 2023-01-24T10:43:37.886 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:18.0.0-1103.g1a404342.el8.x86_64 7/110 2023-01-24T10:43:37.890 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 13/110 2023-01-24T10:43:37.995 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 13/110 2023-01-24T10:43:38.092 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 8/110 2023-01-24T10:43:38.182 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86 14/110 2023-01-24T10:43:38.216 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 8/110 2023-01-24T10:43:38.264 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86 14/110 2023-01-24T10:43:38.409 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 9/110 2023-01-24T10:43:38.432 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 15/110 2023-01-24T10:43:38.495 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 9/110 2023-01-24T10:43:38.571 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 15/110 2023-01-24T10:43:38.703 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:18.0.0-1103.g1a404342.el8.x86_64 10/110 2023-01-24T10:43:38.715 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 16/110 2023-01-24T10:43:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/110 2023-01-24T10:43:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/110 2023-01-24T10:43:39.333 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 16/110 2023-01-24T10:43:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 13/110 2023-01-24T10:43:39.524 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 17/110 2023-01-24T10:43:39.549 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 13/110 2023-01-24T10:43:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86 14/110 2023-01-24T10:43:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86 14/110 2023-01-24T10:43:39.975 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 15/110 2023-01-24T10:43:40.103 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 15/110 2023-01-24T10:43:40.126 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 17/110 2023-01-24T10:43:40.234 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 16/110 2023-01-24T10:43:40.332 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : openblas-0.3.15-3.el8.x86_64 18/110 2023-01-24T10:43:40.847 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 16/110 2023-01-24T10:43:41.004 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 18/110 2023-01-24T10:43:41.033 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 17/110 2023-01-24T10:43:41.331 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 19/110 2023-01-24T10:43:41.527 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 20/110 2023-01-24T10:43:41.613 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 17/110 2023-01-24T10:43:41.704 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 21/110 2023-01-24T10:43:41.841 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.15-3.el8.x86_64 18/110 2023-01-24T10:43:41.937 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/110 2023-01-24T10:43:42.164 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 23/110 2023-01-24T10:43:42.338 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-ceph-common-2:18.0.0-1103.g1a404342.el8. 24/110 2023-01-24T10:43:42.467 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-ceph-argparse-2:18.0.0-1103.g1a404342.el 25/110 2023-01-24T10:43:42.520 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 18/110 2023-01-24T10:43:42.859 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 19/110 2023-01-24T10:43:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 20/110 2023-01-24T10:43:43.230 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 21/110 2023-01-24T10:43:43.329 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cephfs-2:18.0.0-1103.g1a404342.el8.x86_6 26/110 2023-01-24T10:43:43.486 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/110 2023-01-24T10:43:43.661 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 27/110 2023-01-24T10:43:43.708 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 23/110 2023-01-24T10:43:43.881 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:18.0.0-1103.g1a404342.el8. 24/110 2023-01-24T10:43:43.916 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 28/110 2023-01-24T10:43:44.051 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:18.0.0-1103.g1a404342.el 25/110 2023-01-24T10:43:44.174 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/110 2023-01-24T10:43:44.373 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/110 2023-01-24T10:43:44.925 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:18.0.0-1103.g1a404342.el8.x86_6 26/110 2023-01-24T10:43:45.239 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 27/110 2023-01-24T10:43:45.493 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 28/110 2023-01-24T10:43:45.756 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/110 2023-01-24T10:43:45.966 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/110 2023-01-24T10:43:46.129 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/110 2023-01-24T10:43:46.482 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/110 2023-01-24T10:43:46.708 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rsa-4.8-1.el8.noarch 32/110 2023-01-24T10:43:46.900 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 33/110 2023-01-24T10:43:47.070 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rbd-2:18.0.0-1103.g1a404342.el8.x86_64 34/110 2023-01-24T10:43:47.311 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librados-devel-2:18.0.0-1103.g1a404342.el8.x86_6 35/110 2023-01-24T10:43:47.540 INFO:teuthology.orchestra.run.smithi182.stdout: Upgrading : sqlite-3.26.0-16.el8_6.x86_64 36/110 2023-01-24T10:43:47.734 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : fuse-2.9.7-15.el8.x86_64 37/110 2023-01-24T10:43:47.740 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/110 2023-01-24T10:43:47.940 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 38/110 2023-01-24T10:43:48.058 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/110 2023-01-24T10:43:48.311 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.8-1.el8.noarch 32/110 2023-01-24T10:43:48.320 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 39/110 2023-01-24T10:43:48.518 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 33/110 2023-01-24T10:43:48.583 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 40/110 2023-01-24T10:43:48.737 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:18.0.0-1103.g1a404342.el8.x86_64 34/110 2023-01-24T10:43:48.798 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 41/110 2023-01-24T10:43:48.937 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:18.0.0-1103.g1a404342.el8.x86_6 35/110 2023-01-24T10:43:49.041 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 42/110 2023-01-24T10:43:49.150 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : sqlite-3.26.0-16.el8_6.x86_64 36/110 2023-01-24T10:43:49.276 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 43/110 2023-01-24T10:43:49.370 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.7-15.el8.x86_64 37/110 2023-01-24T10:43:49.568 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 38/110 2023-01-24T10:43:49.595 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 44/110 2023-01-24T10:43:49.846 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 45/110 2023-01-24T10:43:49.914 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 39/110 2023-01-24T10:43:50.079 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 46/110 2023-01-24T10:43:50.259 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 40/110 2023-01-24T10:43:50.287 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 47/110 2023-01-24T10:43:50.462 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 48/110 2023-01-24T10:43:50.622 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 49/110 2023-01-24T10:43:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 41/110 2023-01-24T10:43:50.775 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 50/110 2023-01-24T10:43:50.934 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 42/110 2023-01-24T10:43:50.972 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/110 2023-01-24T10:43:51.059 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/110 2023-01-24T10:43:51.180 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 43/110 2023-01-24T10:43:51.441 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : jq-1.6-3.el8.x86_64 52/110 2023-01-24T10:43:51.488 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 44/110 2023-01-24T10:43:51.699 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 53/110 2023-01-24T10:43:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 45/110 2023-01-24T10:43:51.889 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 54/110 2023-01-24T10:43:51.975 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 46/110 2023-01-24T10:43:52.077 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 55/110 2023-01-24T10:43:52.172 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 47/110 2023-01-24T10:43:52.356 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 56/110 2023-01-24T10:43:52.382 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 48/110 2023-01-24T10:43:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 49/110 2023-01-24T10:43:52.595 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 57/110 2023-01-24T10:43:52.727 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 50/110 2023-01-24T10:43:52.785 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 58/110 2023-01-24T10:43:52.898 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/110 2023-01-24T10:43:52.941 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 59/110 2023-01-24T10:43:52.981 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/110 2023-01-24T10:43:53.140 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 60/110 2023-01-24T10:43:53.205 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : jq-1.6-3.el8.x86_64 52/110 2023-01-24T10:43:53.342 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 61/110 2023-01-24T10:43:53.366 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 53/110 2023-01-24T10:43:53.532 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 54/110 2023-01-24T10:43:53.533 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 62/110 2023-01-24T10:43:53.703 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 55/110 2023-01-24T10:43:53.716 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 63/110 2023-01-24T10:43:53.869 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 64/110 2023-01-24T10:43:53.974 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 56/110 2023-01-24T10:43:54.075 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 65/110 2023-01-24T10:43:54.230 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 57/110 2023-01-24T10:43:54.283 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 66/110 2023-01-24T10:43:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 58/110 2023-01-24T10:43:54.502 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 67/110 2023-01-24T10:43:54.534 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 59/110 2023-01-24T10:43:54.759 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 60/110 2023-01-24T10:43:54.767 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-portend-2.6-1.el8.noarch 68/110 2023-01-24T10:43:54.970 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 69/110 2023-01-24T10:43:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 61/110 2023-01-24T10:43:55.128 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 70/110 2023-01-24T10:43:55.193 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 62/110 2023-01-24T10:43:55.344 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 71/110 2023-01-24T10:43:55.359 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 63/110 2023-01-24T10:43:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 64/110 2023-01-24T10:43:55.711 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 65/110 2023-01-24T10:43:55.926 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 66/110 2023-01-24T10:43:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 67/110 2023-01-24T10:43:56.381 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-2.6-1.el8.noarch 68/110 2023-01-24T10:43:56.450 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 72/110 2023-01-24T10:43:56.580 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 69/110 2023-01-24T10:43:56.707 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 73/110 2023-01-24T10:43:56.738 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 70/110 2023-01-24T10:43:56.947 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 71/110 2023-01-24T10:43:56.979 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 74/110 2023-01-24T10:43:57.278 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 75/110 2023-01-24T10:43:57.557 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 76/110 2023-01-24T10:43:57.747 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 77/110 2023-01-24T10:43:58.049 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 72/110 2023-01-24T10:43:58.276 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 73/110 2023-01-24T10:43:58.498 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 78/110 2023-01-24T10:43:58.530 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 74/110 2023-01-24T10:43:58.662 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 79/110 2023-01-24T10:43:58.735 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 79/110 2023-01-24T10:43:58.847 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 75/110 2023-01-24T10:43:58.896 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : python3-rgw-2:18.0.0-1103.g1a404342.el8.x86_64 80/110 2023-01-24T10:43:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 76/110 2023-01-24T10:43:59.372 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 77/110 2023-01-24T10:44:00.126 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 78/110 2023-01-24T10:44:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 79/110 2023-01-24T10:44:00.375 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 79/110 2023-01-24T10:44:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:18.0.0-1103.g1a404342.el8.x86_64 80/110 2023-01-24T10:44:02.638 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 81/110 2023-01-24T10:44:02.862 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 81/110 2023-01-24T10:44:03.437 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 81/110 2023-01-24T10:44:03.702 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 82/110 2023-01-24T10:44:03.787 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 82/110 2023-01-24T10:44:03.945 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 83/110 2023-01-24T10:44:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 81/110 2023-01-24T10:44:04.489 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 81/110 2023-01-24T10:44:05.061 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 81/110 2023-01-24T10:44:05.316 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 82/110 2023-01-24T10:44:05.402 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 82/110 2023-01-24T10:44:05.580 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 83/110 2023-01-24T10:44:23.000 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 83/110 2023-01-24T10:44:23.000 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /sys 2023-01-24T10:44:23.001 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /proc 2023-01-24T10:44:23.001 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /dev 2023-01-24T10:44:23.001 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /run 2023-01-24T10:44:23.001 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /mnt 2023-01-24T10:44:23.002 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /var/tmp 2023-01-24T10:44:23.002 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /home 2023-01-24T10:44:23.002 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /tmp 2023-01-24T10:44:23.002 INFO:teuthology.orchestra.run.smithi182.stdout:skipping the directory /dev 2023-01-24T10:44:23.002 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:23.350 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 84/110 2023-01-24T10:44:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 83/110 2023-01-24T10:44:24.455 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2023-01-24T10:44:24.456 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2023-01-24T10:44:24.456 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /dev 2023-01-24T10:44:24.456 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /run 2023-01-24T10:44:24.456 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2023-01-24T10:44:24.457 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2023-01-24T10:44:24.457 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2023-01-24T10:44:24.457 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2023-01-24T10:44:24.457 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /dev 2023-01-24T10:44:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:24.803 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 84/110 2023-01-24T10:44:24.830 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 84/110 2023-01-24T10:44:24.999 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 85/110 2023-01-24T10:44:25.228 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 85/110 2023-01-24T10:44:25.375 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 86/110 2023-01-24T10:44:25.760 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 86/110 2023-01-24T10:44:25.941 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 87/110 2023-01-24T10:44:25.958 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 87/110 2023-01-24T10:44:26.281 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 84/110 2023-01-24T10:44:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 85/110 2023-01-24T10:44:26.698 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 85/110 2023-01-24T10:44:26.877 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 86/110 2023-01-24T10:44:27.276 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 86/110 2023-01-24T10:44:27.485 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 87/110 2023-01-24T10:44:27.502 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 87/110 2023-01-24T10:44:27.663 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: cephadm-2:18.0.0-1103.g1a404342.el8.noarch 88/110 2023-01-24T10:44:27.851 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : cephadm-2:18.0.0-1103.g1a404342.el8.noarch 88/110 2023-01-24T10:44:27.972 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1103.g1a404342.e 89/110 2023-01-24T10:44:28.144 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1103.g1a404342. 90/110 2023-01-24T10:44:28.349 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noa 91/110 2023-01-24T10:44:28.557 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noa 91/110 2023-01-24T10:44:28.839 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.n 92/110 2023-01-24T10:44:29.182 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:18.0.0-1103.g1a404342.el8.noarch 88/110 2023-01-24T10:44:29.374 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:18.0.0-1103.g1a404342.el8.noarch 88/110 2023-01-24T10:44:29.491 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1103.g1a404342.e 89/110 2023-01-24T10:44:29.680 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1103.g1a404342. 90/110 2023-01-24T10:44:29.800 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.n 92/110 2023-01-24T10:44:29.843 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noa 91/110 2023-01-24T10:44:29.990 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a4 93/110 2023-01-24T10:44:30.053 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noa 91/110 2023-01-24T10:44:30.081 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a4 93/110 2023-01-24T10:44:30.371 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.n 92/110 2023-01-24T10:44:30.432 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1103.g1a404342.el 94/110 2023-01-24T10:44:30.623 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 95/110 2023-01-24T10:44:30.657 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 95/110 2023-01-24T10:44:30.873 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 96/110 2023-01-24T10:44:30.884 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 96/110 2023-01-24T10:44:31.331 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.n 92/110 2023-01-24T10:44:31.535 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a4 93/110 2023-01-24T10:44:31.623 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a4 93/110 2023-01-24T10:44:31.980 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1103.g1a404342.el 94/110 2023-01-24T10:44:32.083 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-2:18.0.0-1103.g1a404342.el8.x86_64 97/110 2023-01-24T10:44:32.188 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 95/110 2023-01-24T10:44:32.223 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 95/110 2023-01-24T10:44:32.299 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 98/110 2023-01-24T10:44:32.333 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 98/110 2023-01-24T10:44:32.401 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 96/110 2023-01-24T10:44:32.412 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 96/110 2023-01-24T10:44:32.498 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1103.g1a404 99/110 2023-01-24T10:44:32.846 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1103.g1a404 99/110 2023-01-24T10:44:32.999 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 100/110 2023-01-24T10:44:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:18.0.0-1103.g1a404342.el8.x86_64 97/110 2023-01-24T10:44:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 98/110 2023-01-24T10:44:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 98/110 2023-01-24T10:44:34.076 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1103.g1a404 99/110 2023-01-24T10:44:34.425 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1103.g1a404 99/110 2023-01-24T10:44:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 100/110 2023-01-24T10:44:37.520 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 100/110 2023-01-24T10:44:37.836 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-test-2:18.0.0-1103.g1a404342.el8.x86_64 101/110 2023-01-24T10:44:38.023 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : ceph-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 102/110 2023-01-24T10:44:38.202 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : sqlite-devel-3.26.0-16.el8_6.x86_64 103/110 2023-01-24T10:44:38.397 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : libcephfs-devel-2:18.0.0-1103.g1a404342.el8.x86_ 104/110 2023-01-24T10:44:38.589 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : rbd-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 105/110 2023-01-24T10:44:38.733 INFO:teuthology.orchestra.run.smithi182.stdout: Installing : rbd-nbd-2:18.0.0-1103.g1a404342.el8.x86_64 106/110 2023-01-24T10:44:38.733 INFO:teuthology.orchestra.run.smithi182.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 107/110 2023-01-24T10:44:38.949 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 107/110 2023-01-24T10:44:39.106 INFO:teuthology.orchestra.run.smithi182.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 108/110 2023-01-24T10:44:39.116 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 100/110 2023-01-24T10:44:39.308 INFO:teuthology.orchestra.run.smithi182.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 109/110 2023-01-24T10:44:39.308 INFO:teuthology.orchestra.run.smithi182.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 110/110 2023-01-24T10:44:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:18.0.0-1103.g1a404342.el8.x86_64 101/110 2023-01-24T10:44:39.605 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 102/110 2023-01-24T10:44:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : sqlite-devel-3.26.0-16.el8_6.x86_64 103/110 2023-01-24T10:44:39.830 INFO:teuthology.orchestra.run.smithi182.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 110/110 2023-01-24T10:44:39.830 INFO:teuthology.orchestra.run.smithi182.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T10:44:39.831 INFO:teuthology.orchestra.run.smithi182.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T10:44:39.831 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:39.942 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:18.0.0-1103.g1a404342.el8.x86_ 104/110 2023-01-24T10:44:40.033 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-2:18.0.0-1103.g1a404342.el8.x86_64 1/110 2023-01-24T10:44:40.033 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 2/110 2023-01-24T10:44:40.033 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 3/110 2023-01-24T10:44:40.034 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 4/110 2023-01-24T10:44:40.034 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1103.g1a404 5/110 2023-01-24T10:44:40.034 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 6/110 2023-01-24T10:44:40.034 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 7/110 2023-01-24T10:44:40.035 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 8/110 2023-01-24T10:44:40.035 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 9/110 2023-01-24T10:44:40.035 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 10/110 2023-01-24T10:44:40.035 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 11/110 2023-01-24T10:44:40.035 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-test-2:18.0.0-1103.g1a404342.el8.x86_64 12/110 2023-01-24T10:44:40.036 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libcephfs-devel-2:18.0.0-1103.g1a404342.el8.x86_ 13/110 2023-01-24T10:44:40.036 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 14/110 2023-01-24T10:44:40.036 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 15/110 2023-01-24T10:44:40.036 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librados-devel-2:18.0.0-1103.g1a404342.el8.x86_6 16/110 2023-01-24T10:44:40.036 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86 17/110 2023-01-24T10:44:40.037 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 18/110 2023-01-24T10:44:40.037 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1103.g1a404342.el 19/110 2023-01-24T10:44:40.037 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-ceph-common-2:18.0.0-1103.g1a404342.el8. 20/110 2023-01-24T10:44:40.037 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cephfs-2:18.0.0-1103.g1a404342.el8.x86_6 21/110 2023-01-24T10:44:40.037 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rados-2:18.0.0-1103.g1a404342.el8.x86_64 22/110 2023-01-24T10:44:40.038 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rbd-2:18.0.0-1103.g1a404342.el8.x86_64 23/110 2023-01-24T10:44:40.038 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rgw-2:18.0.0-1103.g1a404342.el8.x86_64 24/110 2023-01-24T10:44:40.038 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : rbd-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 25/110 2023-01-24T10:44:40.038 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 26/110 2023-01-24T10:44:40.038 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : rbd-nbd-2:18.0.0-1103.g1a404342.el8.x86_64 27/110 2023-01-24T10:44:40.039 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1103.g1a404342. 28/110 2023-01-24T10:44:40.039 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noa 29/110 2023-01-24T10:44:40.039 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.n 30/110 2023-01-24T10:44:40.039 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a4 31/110 2023-01-24T10:44:40.039 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1103.g1a404342.el 32/110 2023-01-24T10:44:40.040 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 33/110 2023-01-24T10:44:40.040 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1103.g1a404342.e 34/110 2023-01-24T10:44:40.040 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 35/110 2023-01-24T10:44:40.040 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : cephadm-2:18.0.0-1103.g1a404342.el8.noarch 36/110 2023-01-24T10:44:40.040 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 37/110 2023-01-24T10:44:40.041 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 38/110 2023-01-24T10:44:40.041 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 39/110 2023-01-24T10:44:40.041 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 40/110 2023-01-24T10:44:40.041 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 41/110 2023-01-24T10:44:40.041 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 42/110 2023-01-24T10:44:40.042 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 43/110 2023-01-24T10:44:40.042 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 44/110 2023-01-24T10:44:40.042 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 45/110 2023-01-24T10:44:40.042 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 46/110 2023-01-24T10:44:40.042 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 47/110 2023-01-24T10:44:40.043 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 48/110 2023-01-24T10:44:40.043 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 49/110 2023-01-24T10:44:40.043 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 50/110 2023-01-24T10:44:40.043 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 51/110 2023-01-24T10:44:40.043 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 52/110 2023-01-24T10:44:40.044 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 53/110 2023-01-24T10:44:40.044 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 54/110 2023-01-24T10:44:40.044 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-portend-2.6-1.el8.noarch 55/110 2023-01-24T10:44:40.044 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 56/110 2023-01-24T10:44:40.045 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 57/110 2023-01-24T10:44:40.045 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 58/110 2023-01-24T10:44:40.045 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 59/110 2023-01-24T10:44:40.045 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 60/110 2023-01-24T10:44:40.045 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 61/110 2023-01-24T10:44:40.045 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 62/110 2023-01-24T10:44:40.046 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 63/110 2023-01-24T10:44:40.046 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 64/110 2023-01-24T10:44:40.046 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 65/110 2023-01-24T10:44:40.046 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 66/110 2023-01-24T10:44:40.047 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 67/110 2023-01-24T10:44:40.047 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 68/110 2023-01-24T10:44:40.047 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 69/110 2023-01-24T10:44:40.047 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 70/110 2023-01-24T10:44:40.047 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 71/110 2023-01-24T10:44:40.048 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 72/110 2023-01-24T10:44:40.048 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 73/110 2023-01-24T10:44:40.048 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 74/110 2023-01-24T10:44:40.048 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 75/110 2023-01-24T10:44:40.049 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 76/110 2023-01-24T10:44:40.049 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 77/110 2023-01-24T10:44:40.049 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 78/110 2023-01-24T10:44:40.049 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 79/110 2023-01-24T10:44:40.049 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 80/110 2023-01-24T10:44:40.050 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 81/110 2023-01-24T10:44:40.050 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 82/110 2023-01-24T10:44:40.050 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 83/110 2023-01-24T10:44:40.050 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 84/110 2023-01-24T10:44:40.050 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 85/110 2023-01-24T10:44:40.051 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 86/110 2023-01-24T10:44:40.051 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/110 2023-01-24T10:44:40.051 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 88/110 2023-01-24T10:44:40.051 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 89/110 2023-01-24T10:44:40.051 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : jq-1.6-3.el8.x86_64 90/110 2023-01-24T10:44:40.052 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/110 2023-01-24T10:44:40.052 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 92/110 2023-01-24T10:44:40.052 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/110 2023-01-24T10:44:40.052 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/110 2023-01-24T10:44:40.052 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 95/110 2023-01-24T10:44:40.053 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 96/110 2023-01-24T10:44:40.053 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 97/110 2023-01-24T10:44:40.053 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 98/110 2023-01-24T10:44:40.053 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 99/110 2023-01-24T10:44:40.053 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 100/110 2023-01-24T10:44:40.054 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 101/110 2023-01-24T10:44:40.054 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : sqlite-devel-3.26.0-16.el8_6.x86_64 102/110 2023-01-24T10:44:40.054 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librados2-2:18.0.0-1103.g1a404342.el8.x86_64 103/110 2023-01-24T10:44:40.054 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 104/110 2023-01-24T10:44:40.054 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 105/110 2023-01-24T10:44:40.055 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 106/110 2023-01-24T10:44:40.055 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : sqlite-3.26.0-16.el8_6.x86_64 107/110 2023-01-24T10:44:40.055 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 108/110 2023-01-24T10:44:40.055 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : sqlite-libs-3.26.0-16.el8_6.x86_64 109/110 2023-01-24T10:44:40.167 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 105/110 2023-01-24T10:44:40.304 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:18.0.0-1103.g1a404342.el8.x86_64 106/110 2023-01-24T10:44:40.305 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 107/110 2023-01-24T10:44:40.477 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 107/110 2023-01-24T10:44:40.676 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : sqlite-3.26.0-15.el8.x86_64 108/110 2023-01-24T10:44:40.844 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : sqlite-libs-3.26.0-15.el8.x86_64 109/110 2023-01-24T10:44:40.845 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 110/110 2023-01-24T10:44:41.346 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 110/110 2023-01-24T10:44:41.347 INFO:teuthology.orchestra.run.smithi078.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T10:44:41.347 INFO:teuthology.orchestra.run.smithi078.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T10:44:41.348 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:41.553 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:18.0.0-1103.g1a404342.el8.x86_64 1/110 2023-01-24T10:44:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 2/110 2023-01-24T10:44:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 3/110 2023-01-24T10:44:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 4/110 2023-01-24T10:44:41.554 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1103.g1a404 5/110 2023-01-24T10:44:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 6/110 2023-01-24T10:44:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 7/110 2023-01-24T10:44:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 8/110 2023-01-24T10:44:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 9/110 2023-01-24T10:44:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 10/110 2023-01-24T10:44:41.555 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 11/110 2023-01-24T10:44:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:18.0.0-1103.g1a404342.el8.x86_64 12/110 2023-01-24T10:44:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:18.0.0-1103.g1a404342.el8.x86_ 13/110 2023-01-24T10:44:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 14/110 2023-01-24T10:44:41.556 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 15/110 2023-01-24T10:44:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:18.0.0-1103.g1a404342.el8.x86_6 16/110 2023-01-24T10:44:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86 17/110 2023-01-24T10:44:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 18/110 2023-01-24T10:44:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1103.g1a404342.el 19/110 2023-01-24T10:44:41.557 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:18.0.0-1103.g1a404342.el8. 20/110 2023-01-24T10:44:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:18.0.0-1103.g1a404342.el8.x86_6 21/110 2023-01-24T10:44:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:18.0.0-1103.g1a404342.el8.x86_64 22/110 2023-01-24T10:44:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:18.0.0-1103.g1a404342.el8.x86_64 23/110 2023-01-24T10:44:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:18.0.0-1103.g1a404342.el8.x86_64 24/110 2023-01-24T10:44:41.558 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 25/110 2023-01-24T10:44:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 26/110 2023-01-24T10:44:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:18.0.0-1103.g1a404342.el8.x86_64 27/110 2023-01-24T10:44:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1103.g1a404342. 28/110 2023-01-24T10:44:41.559 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noa 29/110 2023-01-24T10:44:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.n 30/110 2023-01-24T10:44:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a4 31/110 2023-01-24T10:44:41.560 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1103.g1a404342.el 32/110 2023-01-24T10:44:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 33/110 2023-01-24T10:44:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1103.g1a404342.e 34/110 2023-01-24T10:44:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 35/110 2023-01-24T10:44:41.561 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:18.0.0-1103.g1a404342.el8.noarch 36/110 2023-01-24T10:44:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 37/110 2023-01-24T10:44:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 38/110 2023-01-24T10:44:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 39/110 2023-01-24T10:44:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 40/110 2023-01-24T10:44:41.562 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 41/110 2023-01-24T10:44:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 42/110 2023-01-24T10:44:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 43/110 2023-01-24T10:44:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 44/110 2023-01-24T10:44:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 45/110 2023-01-24T10:44:41.563 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 46/110 2023-01-24T10:44:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 47/110 2023-01-24T10:44:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 48/110 2023-01-24T10:44:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 49/110 2023-01-24T10:44:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 50/110 2023-01-24T10:44:41.564 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 51/110 2023-01-24T10:44:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 52/110 2023-01-24T10:44:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 53/110 2023-01-24T10:44:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 54/110 2023-01-24T10:44:41.565 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-2.6-1.el8.noarch 55/110 2023-01-24T10:44:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 56/110 2023-01-24T10:44:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 57/110 2023-01-24T10:44:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 58/110 2023-01-24T10:44:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 59/110 2023-01-24T10:44:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 60/110 2023-01-24T10:44:41.566 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 61/110 2023-01-24T10:44:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 62/110 2023-01-24T10:44:41.567 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 63/110 2023-01-24T10:44:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 64/110 2023-01-24T10:44:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 65/110 2023-01-24T10:44:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 66/110 2023-01-24T10:44:41.568 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 67/110 2023-01-24T10:44:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 68/110 2023-01-24T10:44:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 69/110 2023-01-24T10:44:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 70/110 2023-01-24T10:44:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 71/110 2023-01-24T10:44:41.569 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 72/110 2023-01-24T10:44:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 73/110 2023-01-24T10:44:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 74/110 2023-01-24T10:44:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 75/110 2023-01-24T10:44:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 76/110 2023-01-24T10:44:41.570 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 77/110 2023-01-24T10:44:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 78/110 2023-01-24T10:44:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 79/110 2023-01-24T10:44:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 80/110 2023-01-24T10:44:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 81/110 2023-01-24T10:44:41.571 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 82/110 2023-01-24T10:44:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 83/110 2023-01-24T10:44:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 84/110 2023-01-24T10:44:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 85/110 2023-01-24T10:44:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 86/110 2023-01-24T10:44:41.572 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/110 2023-01-24T10:44:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 88/110 2023-01-24T10:44:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 89/110 2023-01-24T10:44:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : jq-1.6-3.el8.x86_64 90/110 2023-01-24T10:44:41.573 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 91/110 2023-01-24T10:44:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 92/110 2023-01-24T10:44:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 93/110 2023-01-24T10:44:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 94/110 2023-01-24T10:44:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 95/110 2023-01-24T10:44:41.574 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 96/110 2023-01-24T10:44:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 97/110 2023-01-24T10:44:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 98/110 2023-01-24T10:44:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 99/110 2023-01-24T10:44:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 100/110 2023-01-24T10:44:41.575 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 101/110 2023-01-24T10:44:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : sqlite-devel-3.26.0-16.el8_6.x86_64 102/110 2023-01-24T10:44:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:18.0.0-1103.g1a404342.el8.x86_64 103/110 2023-01-24T10:44:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 104/110 2023-01-24T10:44:41.576 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 105/110 2023-01-24T10:44:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 106/110 2023-01-24T10:44:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : sqlite-3.26.0-16.el8_6.x86_64 107/110 2023-01-24T10:44:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : sqlite-3.26.0-15.el8.x86_64 108/110 2023-01-24T10:44:41.577 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : sqlite-libs-3.26.0-16.el8_6.x86_64 109/110 2023-01-24T10:44:42.469 INFO:teuthology.orchestra.run.smithi182.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 110/110 2023-01-24T10:44:42.469 INFO:teuthology.orchestra.run.smithi182.stdout:Last metadata expiration check: 0:01:37 ago on Tue 24 Jan 2023 10:43:05 AM UTC. 2023-01-24T10:44:44.057 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : sqlite-libs-3.26.0-15.el8.x86_64 110/110 2023-01-24T10:44:44.058 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:01:38 ago on Tue 24 Jan 2023 10:43:06 AM UTC. 2023-01-24T10:44:44.824 INFO:teuthology.orchestra.run.smithi182.stdout:Installed products updated. 2023-01-24T10:44:45.039 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:45.040 INFO:teuthology.orchestra.run.smithi182.stdout:Upgraded: 2023-01-24T10:44:45.040 INFO:teuthology.orchestra.run.smithi182.stdout: librados2-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.040 INFO:teuthology.orchestra.run.smithi182.stdout: librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.040 INFO:teuthology.orchestra.run.smithi182.stdout: sqlite-3.26.0-16.el8_6.x86_64 2023-01-24T10:44:45.041 INFO:teuthology.orchestra.run.smithi182.stdout: sqlite-libs-3.26.0-16.el8_6.x86_64 2023-01-24T10:44:45.041 INFO:teuthology.orchestra.run.smithi182.stdout:Installed: 2023-01-24T10:44:45.041 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.041 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.041 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.042 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.042 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-grafana-dashboards-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.042 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-immutable-object-cache-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.042 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.043 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.043 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.043 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.043 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.043 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-modules-core-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.044 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.044 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.044 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.044 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-prometheus-alerts-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.044 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.045 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.045 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-test-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.045 INFO:teuthology.orchestra.run.smithi182.stdout: ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.045 INFO:teuthology.orchestra.run.smithi182.stdout: cephadm-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:45.046 INFO:teuthology.orchestra.run.smithi182.stdout: fuse-2.9.7-15.el8.x86_64 2023-01-24T10:44:45.046 INFO:teuthology.orchestra.run.smithi182.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T10:44:45.046 INFO:teuthology.orchestra.run.smithi182.stdout: jq-1.6-3.el8.x86_64 2023-01-24T10:44:45.046 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs-devel-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.047 INFO:teuthology.orchestra.run.smithi182.stdout: libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.047 INFO:teuthology.orchestra.run.smithi182.stdout: libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.047 INFO:teuthology.orchestra.run.smithi182.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-24T10:44:45.048 INFO:teuthology.orchestra.run.smithi182.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T10:44:45.048 INFO:teuthology.orchestra.run.smithi182.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-24T10:44:45.049 INFO:teuthology.orchestra.run.smithi182.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T10:44:45.049 INFO:teuthology.orchestra.run.smithi182.stdout: librados-devel-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.049 INFO:teuthology.orchestra.run.smithi182.stdout: libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.049 INFO:teuthology.orchestra.run.smithi182.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-24T10:44:45.049 INFO:teuthology.orchestra.run.smithi182.stdout: librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.050 INFO:teuthology.orchestra.run.smithi182.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T10:44:45.050 INFO:teuthology.orchestra.run.smithi182.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T10:44:45.050 INFO:teuthology.orchestra.run.smithi182.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T10:44:45.050 INFO:teuthology.orchestra.run.smithi182.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-24T10:44:45.050 INFO:teuthology.orchestra.run.smithi182.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-24T10:44:45.051 INFO:teuthology.orchestra.run.smithi182.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T10:44:45.051 INFO:teuthology.orchestra.run.smithi182.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-24T10:44:45.051 INFO:teuthology.orchestra.run.smithi182.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T10:44:45.051 INFO:teuthology.orchestra.run.smithi182.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T10:44:45.051 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T10:44:45.052 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-argparse-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.052 INFO:teuthology.orchestra.run.smithi182.stdout: python3-ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.052 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cephfs-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.052 INFO:teuthology.orchestra.run.smithi182.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T10:44:45.052 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-24T10:44:45.053 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T10:44:45.053 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T10:44:45.053 INFO:teuthology.orchestra.run.smithi182.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-24T10:44:45.054 INFO:teuthology.orchestra.run.smithi182.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T10:44:45.054 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T10:44:45.054 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T10:44:45.054 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-24T10:44:45.054 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T10:44:45.055 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T10:44:45.055 INFO:teuthology.orchestra.run.smithi182.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T10:44:45.055 INFO:teuthology.orchestra.run.smithi182.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T10:44:45.055 INFO:teuthology.orchestra.run.smithi182.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T10:44:45.055 INFO:teuthology.orchestra.run.smithi182.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T10:44:45.056 INFO:teuthology.orchestra.run.smithi182.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T10:44:45.056 INFO:teuthology.orchestra.run.smithi182.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T10:44:45.056 INFO:teuthology.orchestra.run.smithi182.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T10:44:45.056 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-24T10:44:45.056 INFO:teuthology.orchestra.run.smithi182.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-24T10:44:45.057 INFO:teuthology.orchestra.run.smithi182.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T10:44:45.057 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T10:44:45.057 INFO:teuthology.orchestra.run.smithi182.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T10:44:45.057 INFO:teuthology.orchestra.run.smithi182.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T10:44:45.057 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-24T10:44:45.058 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T10:44:45.058 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T10:44:45.058 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-24T10:44:45.058 INFO:teuthology.orchestra.run.smithi182.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T10:44:45.058 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rados-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.058 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rbd-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.059 INFO:teuthology.orchestra.run.smithi182.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T10:44:45.059 INFO:teuthology.orchestra.run.smithi182.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T10:44:45.059 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rgw-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.059 INFO:teuthology.orchestra.run.smithi182.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T10:44:45.059 INFO:teuthology.orchestra.run.smithi182.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T10:44:45.060 INFO:teuthology.orchestra.run.smithi182.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-24T10:44:45.060 INFO:teuthology.orchestra.run.smithi182.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T10:44:45.060 INFO:teuthology.orchestra.run.smithi182.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T10:44:45.060 INFO:teuthology.orchestra.run.smithi182.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T10:44:45.060 INFO:teuthology.orchestra.run.smithi182.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T10:44:45.061 INFO:teuthology.orchestra.run.smithi182.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T10:44:45.061 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T10:44:45.061 INFO:teuthology.orchestra.run.smithi182.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T10:44:45.061 INFO:teuthology.orchestra.run.smithi182.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T10:44:45.061 INFO:teuthology.orchestra.run.smithi182.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T10:44:45.062 INFO:teuthology.orchestra.run.smithi182.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T10:44:45.062 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.062 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.062 INFO:teuthology.orchestra.run.smithi182.stdout: rbd-nbd-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:45.063 INFO:teuthology.orchestra.run.smithi182.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-24T10:44:45.063 INFO:teuthology.orchestra.run.smithi182.stdout: sqlite-devel-3.26.0-16.el8_6.x86_64 2023-01-24T10:44:45.064 INFO:teuthology.orchestra.run.smithi182.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T10:44:45.064 INFO:teuthology.orchestra.run.smithi182.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T10:44:45.064 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:45.065 INFO:teuthology.orchestra.run.smithi182.stdout:Complete! 2023-01-24T10:44:45.205 DEBUG:teuthology.parallel:result is None 2023-01-24T10:44:46.398 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2023-01-24T10:44:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:46.610 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2023-01-24T10:44:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: sqlite-3.26.0-16.el8_6.x86_64 2023-01-24T10:44:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: sqlite-libs-3.26.0-16.el8_6.x86_64 2023-01-24T10:44:46.611 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2023-01-24T10:44:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:18.0.0-1103.g1a404342.el8.noarch 2023-01-24T10:44:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: fuse-2.9.7-15.el8.x86_64 2023-01-24T10:44:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T10:44:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: jq-1.6-3.el8.x86_64 2023-01-24T10:44:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-24T10:44:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T10:44:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-24T10:44:46.617 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T10:44:46.617 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.617 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.617 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-24T10:44:46.617 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.617 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T10:44:46.618 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T10:44:46.618 INFO:teuthology.orchestra.run.smithi078.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T10:44:46.618 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-24T10:44:46.618 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-24T10:44:46.618 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T10:44:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-24T10:44:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T10:44:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T10:44:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T10:44:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.619 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T10:44:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-24T10:44:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T10:44:46.620 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T10:44:46.621 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-24T10:44:46.621 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T10:44:46.621 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T10:44:46.621 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T10:44:46.621 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-24T10:44:46.622 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T10:44:46.622 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T10:44:46.622 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T10:44:46.622 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T10:44:46.622 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T10:44:46.623 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T10:44:46.623 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T10:44:46.623 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T10:44:46.623 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T10:44:46.623 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-24T10:44:46.624 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-24T10:44:46.624 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T10:44:46.624 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T10:44:46.624 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T10:44:46.624 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T10:44:46.625 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-24T10:44:46.625 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T10:44:46.625 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T10:44:46.625 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-24T10:44:46.625 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T10:44:46.625 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.626 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.626 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T10:44:46.626 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T10:44:46.626 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.626 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T10:44:46.627 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T10:44:46.627 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-24T10:44:46.627 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T10:44:46.627 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T10:44:46.627 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T10:44:46.628 INFO:teuthology.orchestra.run.smithi078.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T10:44:46.628 INFO:teuthology.orchestra.run.smithi078.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T10:44:46.628 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T10:44:46.628 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T10:44:46.628 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T10:44:46.628 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T10:44:46.629 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T10:44:46.629 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.629 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.629 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:18.0.0-1103.g1a404342.el8.x86_64 2023-01-24T10:44:46.629 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-24T10:44:46.630 INFO:teuthology.orchestra.run.smithi078.stdout: sqlite-devel-3.26.0-16.el8_6.x86_64 2023-01-24T10:44:46.630 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T10:44:46.630 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T10:44:46.631 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:46.631 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2023-01-24T10:44:46.759 DEBUG:teuthology.parallel:result is None 2023-01-24T10:44:46.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo lsb_release -is 2023-01-24T10:44:46.814 INFO:teuthology.orchestra.run.smithi078.stdout:RedHatEnterprise 2023-01-24T10:44:46.815 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T10:44:46.816 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1a4043421bb637c432ee02d001db3c6ec52c48e0 2023-01-24T10:44:47.010 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T10:44:47.033 INFO:teuthology.orchestra.run.smithi078.stdout:18.0.0-1103.g1a404342.el8 2023-01-24T10:44:47.033 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1103.g1a404342.el8 2023-01-24T10:44:47.034 INFO:teuthology.task.install:The correct ceph version 18.0.0-1103.g1a404342 is installed. 2023-01-24T10:44:47.034 DEBUG:teuthology.orchestra.run.smithi182:> sudo lsb_release -is 2023-01-24T10:44:47.099 INFO:teuthology.orchestra.run.smithi182.stdout:RedHatEnterprise 2023-01-24T10:44:47.100 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T10:44:47.101 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1a4043421bb637c432ee02d001db3c6ec52c48e0 2023-01-24T10:44:47.262 DEBUG:teuthology.orchestra.run.smithi182:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T10:44:47.286 INFO:teuthology.orchestra.run.smithi182.stdout:18.0.0-1103.g1a404342.el8 2023-01-24T10:44:47.287 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1103.g1a404342.el8 2023-01-24T10:44:47.287 INFO:teuthology.task.install:The correct ceph version 18.0.0-1103.g1a404342 is installed. 2023-01-24T10:44:47.287 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T10:44:47.288 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:47.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T10:44:47.315 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:44:47.316 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T10:44:47.354 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T10:44:47.355 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:47.355 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T10:44:47.381 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T10:44:47.448 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:44:47.448 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T10:44:47.476 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T10:44:47.543 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T10:44:47.544 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:47.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T10:44:47.571 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T10:44:47.637 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:44:47.637 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T10:44:47.665 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T10:44:47.731 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T10:44:47.794 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T10:44:47.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /var/log/ceph 2023-01-24T10:44:47.797 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod 777 /var/log/ceph 2023-01-24T10:44:47.824 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T10:44:47.824 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T10:44:47.864 DEBUG:teuthology.orchestra.run.smithi182:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T10:44:47.891 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T10:44:47.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T10:44:47.930 DEBUG:teuthology.orchestra.run.smithi182:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T10:44:47.969 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T10:44:47.970 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_class_default_list': '*', 'osd_class_load_list': '*'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(PG_AVAILABILITY\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-24T10:44:47.970 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi/7134893', 'branch': 'wip-kcheda3-multisite-20230112', 'description': 'rados:basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/rados_cls_all}', 'email': None, 'first_in_suite': False, 'job_id': '7134893', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'bcs-ceph-2023-01-24_07:57:38-rados:basic-wip-kcheda3-multisite-20230112-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-kcheda3-multisite-20230112'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_class_default_list': '*', 'osd_class_load_list': '*'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(PG_AVAILABILITY\\)'], 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-kcheda3-multisite-20230112', 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0'}}, 'owner': 'scheduled_bcs-ceph@teuthology', 'priority': 300, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0', 'sleep_before_teardown': 0, 'suite': 'rados:basic', 'suite_branch': 'wip-kcheda3-multisite-20230112', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1a4043421bb637c432ee02d001db3c6ec52c48e0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0', 'targets': {'smithi078.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi182.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_system_packages': {'deb': ['sqlite3'], 'rpm': ['sqlite-devel']}, 'flavor': 'default', 'sha1': '1a4043421bb637c432ee02d001db3c6ec52c48e0'}}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['cls']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_07:57:38', 'tube': 'smithi', 'user': 'bcs-ceph', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T10:44:47.971 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T10:44:47.997 DEBUG:teuthology.orchestra.run.smithi182:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T10:44:48.015 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T10:44:48.054 DEBUG:teuthology.orchestra.run.smithi182:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T10:44:48.080 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:48.081 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T10:44:48.136 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T10:44:48.137 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2023-01-24T10:44:48.194 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T10:44:48.194 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:48.195 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 351597 Links: 1 2023-01-24T10:44:48.195 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:48.195 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:48.195 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-01-24 10:44:41.263882771 +0000 2023-01-24T10:44:48.196 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-01-24 10:42:11.772864922 +0000 2023-01-24T10:44:48.196 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-01-24 10:42:11.772864922 +0000 2023-01-24T10:44:48.196 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-01-24T10:44:48.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T10:44:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-01-24T10:44:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-01-24T10:44:48.260 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000266104 s, 1.9 MB/s 2023-01-24T10:44:48.261 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T10:44:48.320 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2023-01-24T10:44:48.380 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T10:44:48.380 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:48.381 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 355680 Links: 1 2023-01-24T10:44:48.381 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:48.381 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:48.381 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-01-24 10:44:41.263882771 +0000 2023-01-24T10:44:48.382 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-01-24 10:42:12.266855870 +0000 2023-01-24T10:44:48.382 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-01-24 10:42:12.266855870 +0000 2023-01-24T10:44:48.382 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-01-24T10:44:48.383 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T10:44:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-01-24T10:44:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-01-24T10:44:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000211013 s, 2.4 MB/s 2023-01-24T10:44:48.447 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T10:44:48.504 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2023-01-24T10:44:48.560 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T10:44:48.560 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:48.560 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 360464 Links: 1 2023-01-24T10:44:48.561 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:48.561 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:48.561 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-01-24 10:44:41.264882753 +0000 2023-01-24T10:44:48.561 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-01-24 10:42:12.772846597 +0000 2023-01-24T10:44:48.561 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-01-24 10:42:12.772846597 +0000 2023-01-24T10:44:48.562 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-01-24T10:44:48.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T10:44:48.625 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-01-24T10:44:48.625 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-01-24T10:44:48.625 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000294412 s, 1.7 MB/s 2023-01-24T10:44:48.626 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T10:44:48.684 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2023-01-24T10:44:48.741 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T10:44:48.741 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:48.741 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 360565 Links: 1 2023-01-24T10:44:48.742 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:48.742 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:48.742 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2023-01-24 10:44:41.264882753 +0000 2023-01-24T10:44:48.742 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2023-01-24 10:42:13.261837636 +0000 2023-01-24T10:44:48.742 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2023-01-24 10:42:13.261837636 +0000 2023-01-24T10:44:48.743 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2023-01-24T10:44:48.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T10:44:48.806 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2023-01-24T10:44:48.806 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2023-01-24T10:44:48.807 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000246706 s, 2.1 MB/s 2023-01-24T10:44:48.808 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T10:44:48.865 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T10:44:48.865 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:44:48.866 DEBUG:teuthology.orchestra.run.smithi182:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T10:44:48.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T10:44:48.885 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_1 2023-01-24T10:44:48.943 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T10:44:48.943 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:48.944 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 356456 Links: 1 2023-01-24T10:44:48.944 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:48.944 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:48.944 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-01-24 10:44:39.751427561 +0000 2023-01-24T10:44:48.945 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-01-24 10:42:19.698712609 +0000 2023-01-24T10:44:48.945 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-01-24 10:42:19.698712609 +0000 2023-01-24T10:44:48.945 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-01-24T10:44:48.945 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T10:44:49.011 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-01-24T10:44:49.012 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-01-24T10:44:49.012 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.000261304 s, 2.0 MB/s 2023-01-24T10:44:49.013 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T10:44:49.073 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_2 2023-01-24T10:44:49.131 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T10:44:49.131 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:49.132 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 356619 Links: 1 2023-01-24T10:44:49.132 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:49.132 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:49.132 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-01-24 10:44:39.751427561 +0000 2023-01-24T10:44:49.132 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-01-24 10:42:20.202700621 +0000 2023-01-24T10:44:49.133 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-01-24 10:42:20.202700621 +0000 2023-01-24T10:44:49.133 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-01-24T10:44:49.133 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T10:44:49.198 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-01-24T10:44:49.199 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-01-24T10:44:49.199 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.00032478 s, 1.6 MB/s 2023-01-24T10:44:49.200 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T10:44:49.259 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_3 2023-01-24T10:44:49.317 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T10:44:49.317 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:49.317 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 356649 Links: 1 2023-01-24T10:44:49.318 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:49.318 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:49.318 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-01-24 10:44:39.752427538 +0000 2023-01-24T10:44:49.318 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-01-24 10:42:20.687689084 +0000 2023-01-24T10:44:49.318 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-01-24 10:42:20.687689084 +0000 2023-01-24T10:44:49.319 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-01-24T10:44:49.319 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T10:44:49.382 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-01-24T10:44:49.382 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-01-24T10:44:49.383 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.000256088 s, 2.0 MB/s 2023-01-24T10:44:49.384 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T10:44:49.442 DEBUG:teuthology.orchestra.run.smithi182:> stat /dev/vg_nvme/lv_4 2023-01-24T10:44:49.500 INFO:teuthology.orchestra.run.smithi182.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T10:44:49.500 INFO:teuthology.orchestra.run.smithi182.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T10:44:49.501 INFO:teuthology.orchestra.run.smithi182.stdout:Device: 6h/6d Inode: 351210 Links: 1 2023-01-24T10:44:49.501 INFO:teuthology.orchestra.run.smithi182.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T10:44:49.501 INFO:teuthology.orchestra.run.smithi182.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T10:44:49.501 INFO:teuthology.orchestra.run.smithi182.stdout:Access: 2023-01-24 10:44:39.752427538 +0000 2023-01-24T10:44:49.502 INFO:teuthology.orchestra.run.smithi182.stdout:Modify: 2023-01-24 10:42:21.174677499 +0000 2023-01-24T10:44:49.502 INFO:teuthology.orchestra.run.smithi182.stdout:Change: 2023-01-24 10:42:21.174677499 +0000 2023-01-24T10:44:49.502 INFO:teuthology.orchestra.run.smithi182.stdout: Birth: - 2023-01-24T10:44:49.502 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T10:44:49.565 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records in 2023-01-24T10:44:49.566 INFO:teuthology.orchestra.run.smithi182.stderr:1+0 records out 2023-01-24T10:44:49.566 INFO:teuthology.orchestra.run.smithi182.stderr:512 bytes copied, 0.000175607 s, 2.9 MB/s 2023-01-24T10:44:49.567 DEBUG:teuthology.orchestra.run.smithi182:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T10:44:49.625 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T10:44:49.626 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi182.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T10:44:49.626 INFO:tasks.ceph:Generating config... 2023-01-24T10:44:49.628 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-24T10:44:49.628 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-01-24T10:44:49.629 INFO:tasks.ceph:[global] ms bind msgr1 = False 2023-01-24T10:44:49.629 INFO:tasks.ceph:[global] ms bind msgr2 = True 2023-01-24T10:44:49.629 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-01-24T10:44:49.629 INFO:tasks.ceph:[global] ms type = async 2023-01-24T10:44:49.630 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T10:44:49.630 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T10:44:49.630 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T10:44:49.630 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T10:44:49.631 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T10:44:49.631 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-24T10:44:49.631 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-24T10:44:49.631 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-24T10:44:49.632 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-24T10:44:49.632 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-24T10:44:49.632 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-24T10:44:49.632 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-24T10:44:49.633 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2023-01-24T10:44:49.633 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-01-24T10:44:49.633 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-24T10:44:49.634 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-24T10:44:49.634 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-24T10:44:49.634 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-24T10:44:49.634 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T10:44:49.635 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T10:44:49.635 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-24T10:44:49.635 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-24T10:44:49.636 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-24T10:44:49.636 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-24T10:44:49.636 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-24T10:44:49.637 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-24T10:44:49.637 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-24T10:44:49.637 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-24T10:44:49.637 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-24T10:44:49.638 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-24T10:44:49.638 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-24T10:44:49.638 INFO:tasks.ceph:[osd] osd_class_default_list = * 2023-01-24T10:44:49.639 INFO:tasks.ceph:[osd] osd_class_load_list = * 2023-01-24T10:44:49.639 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T10:44:49.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T10:44:49.721 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T10:44:49.725 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T10:44:49.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T10:44:49.813 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.78'), ('mon.c', '[v2:172.21.15.78:3301,v1:172.21.15.78:6790]'), ('mon.b', '172.21.15.182')] 2023-01-24T10:44:49.814 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.78,[v2:172.21.15.78:3301,v1:172.21.15.78:6790],172.21.15.182', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_class_default_list': '*', 'osd_class_load_list': '*'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-24T10:44:49.816 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:49.816 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T10:44:49.874 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.78 --addv c '[v2:172.21.15.78:3301,v1:172.21.15.78:6790]' --add b 172.21.15.182 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:44:49.968 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:44:49.969 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: generated fsid 74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12 2023-01-24T10:44:49.969 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = pacific 2023-01-24T10:44:49.969 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 0 2023-01-24T10:44:49.969 INFO:teuthology.orchestra.run.smithi078.stdout:fsid 74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12 2023-01-24T10:44:49.969 INFO:teuthology.orchestra.run.smithi078.stdout:last_changed 2023-01-24T10:44:49.966946+0000 2023-01-24T10:44:49.970 INFO:teuthology.orchestra.run.smithi078.stdout:created 2023-01-24T10:44:49.966946+0000 2023-01-24T10:44:49.970 INFO:teuthology.orchestra.run.smithi078.stdout:min_mon_release 16 (pacific) 2023-01-24T10:44:49.970 INFO:teuthology.orchestra.run.smithi078.stdout:election_strategy: 1 2023-01-24T10:44:49.970 INFO:teuthology.orchestra.run.smithi078.stdout:0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2023-01-24T10:44:49.970 INFO:teuthology.orchestra.run.smithi078.stdout:1: [v2:172.21.15.182:3300/0,v1:172.21.15.182:6789/0] mon.b 2023-01-24T10:44:49.971 INFO:teuthology.orchestra.run.smithi078.stdout:2: [v2:172.21.15.78:3301/0,v1:172.21.15.78:6790/0] mon.c 2023-01-24T10:44:49.971 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-24T10:44:49.972 DEBUG:teuthology.orchestra.run.smithi078:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T10:44:49.990 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12... 2023-01-24T10:44:49.991 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T10:44:50.033 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T10:44:50.079 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2023-01-24T10:44:50.079 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2023-01-24T10:44:50.080 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T10:44:50.080 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2023-01-24T10:44:50.080 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.080 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2023-01-24T10:44:50.080 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.082 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2023-01-24T10:44:50.083 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.083 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2023-01-24T10:44:50.083 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2023-01-24T10:44:50.083 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.084 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2023-01-24T10:44:50.084 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2023-01-24T10:44:50.084 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.084 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T10:44:50.084 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2023-01-24T10:44:50.085 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.085 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T10:44:50.085 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T10:44:50.085 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.085 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2023-01-24T10:44:50.086 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.088 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2023-01-24T10:44:50.088 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2023-01-24T10:44:50.088 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2023-01-24T10:44:50.089 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2023-01-24T10:44:50.089 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2023-01-24T10:44:50.089 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2023-01-24T10:44:50.089 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2023-01-24T10:44:50.089 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T10:44:50.090 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T10:44:50.090 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2023-01-24T10:44:50.090 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.090 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T10:44:50.090 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.091 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2023-01-24T10:44:50.091 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.091 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2023-01-24T10:44:50.091 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.091 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2023-01-24T10:44:50.091 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2023-01-24T10:44:50.092 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2023-01-24T10:44:50.092 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,[v2:172.21.15.78:3301,v1:172.21.15.78:6790],172.21.15.182" 2023-01-24T10:44:50.092 INFO:teuthology.orchestra.run.smithi078.stdout: mon client directed command retry = 5 2023-01-24T10:44:50.092 INFO:teuthology.orchestra.run.smithi078.stdout: mon election default strategy = 1 2023-01-24T10:44:50.093 INFO:teuthology.orchestra.run.smithi078.stdout: ms bind msgr1 = False 2023-01-24T10:44:50.093 INFO:teuthology.orchestra.run.smithi078.stdout: ms bind msgr2 = True 2023-01-24T10:44:50.093 INFO:teuthology.orchestra.run.smithi078.stdout: ms inject socket failures = 5000 2023-01-24T10:44:50.093 INFO:teuthology.orchestra.run.smithi078.stdout: ms type = async 2023-01-24T10:44:50.093 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = 74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12 2023-01-24T10:44:50.094 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.094 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2023-01-24T10:44:50.094 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2023-01-24T10:44:50.094 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.094 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2023-01-24T10:44:50.094 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2023-01-24T10:44:50.095 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.095 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2023-01-24T10:44:50.095 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2023-01-24T10:44:50.095 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.095 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2023-01-24T10:44:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2023-01-24T10:44:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2023-01-24T10:44:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2023-01-24T10:44:50.096 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2023-01-24T10:44:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2023-01-24T10:44:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2023-01-24T10:44:50.097 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2023-01-24T10:44:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2023-01-24T10:44:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2023-01-24T10:44:50.098 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2023-01-24T10:44:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2023-01-24T10:44:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2023-01-24T10:44:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore compression algorithm = snappy 2023-01-24T10:44:50.099 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore compression mode = aggressive 2023-01-24T10:44:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2023-01-24T10:44:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore zero block detection = True 2023-01-24T10:44:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 20 2023-01-24T10:44:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 20 2023-01-24T10:44:50.100 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-01-24T10:44:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2023-01-24T10:44:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 10 2023-01-24T10:44:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T10:44:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2023-01-24T10:44:50.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2023-01-24T10:44:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify cached snaps = True 2023-01-24T10:44:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify missing on start = True 2023-01-24T10:44:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2023-01-24T10:44:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2023-01-24T10:44:50.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd op queue = debug_random 2023-01-24T10:44:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd op queue cut off = debug_random 2023-01-24T10:44:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd shutdown pgref assert = True 2023-01-24T10:44:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd_class_default_list = * 2023-01-24T10:44:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd_class_load_list = * 2023-01-24T10:44:50.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.103 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2023-01-24T10:44:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-01-24T10:44:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2023-01-24T10:44:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-01-24T10:44:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-01-24T10:44:50.104 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-01-24T10:44:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-01-24T10:44:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2023-01-24T10:44:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.105 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2023-01-24T10:44:50.105 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2023-01-24T10:44:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2023-01-24T10:44:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2023-01-24T10:44:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2023-01-24T10:44:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2023-01-24T10:44:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2023-01-24T10:44:50.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2023-01-24T10:44:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2023-01-24T10:44:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2023-01-24T10:44:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2023-01-24T10:44:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.107 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T10:44:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T10:44:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T10:44:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T10:44:50.108 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T10:44:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T10:44:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2023-01-24T10:44:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2023-01-24T10:44:50.109 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.110 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2023-01-24T10:44:50.110 INFO:teuthology.orchestra.run.smithi078.stdout: mon min osdmap epochs = 50 2023-01-24T10:44:50.110 INFO:teuthology.orchestra.run.smithi078.stdout: mon osdmap full prune interval = 2 2023-01-24T10:44:50.110 INFO:teuthology.orchestra.run.smithi078.stdout: mon osdmap full prune min = 15 2023-01-24T10:44:50.110 INFO:teuthology.orchestra.run.smithi078.stdout: mon osdmap full prune txsize = 2 2023-01-24T10:44:50.111 INFO:teuthology.orchestra.run.smithi078.stdout: mon scrub interval = 300 2023-01-24T10:44:50.111 INFO:teuthology.orchestra.run.smithi078.stdout: paxos service trim min = 10 2023-01-24T10:44:50.111 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:44:50.111 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2023-01-24T10:44:50.111 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2023-01-24T10:44:50.112 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2023-01-24T10:44:50.112 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2023-01-24T10:44:50.112 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T10:44:50.112 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T10:44:50.112 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2023-01-24T10:44:50.113 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2023-01-24T10:44:50.113 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2023-01-24T10:44:50.115 INFO:teuthology.orchestra.run.smithi182.stdout:[global] 2023-01-24T10:44:50.115 INFO:teuthology.orchestra.run.smithi182.stdout: chdir = "" 2023-01-24T10:44:50.115 INFO:teuthology.orchestra.run.smithi182.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T10:44:50.115 INFO:teuthology.orchestra.run.smithi182.stdout: auth supported = cephx 2023-01-24T10:44:50.115 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.116 INFO:teuthology.orchestra.run.smithi182.stdout: filestore xattr use omap = true 2023-01-24T10:44:50.116 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.116 INFO:teuthology.orchestra.run.smithi182.stdout: mon clock drift allowed = 1.000 2023-01-24T10:44:50.116 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.116 INFO:teuthology.orchestra.run.smithi182.stdout: osd crush chooseleaf type = 0 2023-01-24T10:44:50.117 INFO:teuthology.orchestra.run.smithi182.stdout: auth debug = true 2023-01-24T10:44:50.117 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.117 INFO:teuthology.orchestra.run.smithi182.stdout: ms die on old message = true 2023-01-24T10:44:50.117 INFO:teuthology.orchestra.run.smithi182.stdout: ms die on bug = true 2023-01-24T10:44:50.117 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.118 INFO:teuthology.orchestra.run.smithi182.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T10:44:50.118 INFO:teuthology.orchestra.run.smithi182.stdout: mon pg warn max object skew = 0 2023-01-24T10:44:50.118 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.118 INFO:teuthology.orchestra.run.smithi182.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T10:44:50.118 INFO:teuthology.orchestra.run.smithi182.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T10:44:50.119 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.119 INFO:teuthology.orchestra.run.smithi182.stdout: osd pool default size = 2 2023-01-24T10:44:50.119 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.119 INFO:teuthology.orchestra.run.smithi182.stdout: mon osd allow primary affinity = true 2023-01-24T10:44:50.119 INFO:teuthology.orchestra.run.smithi182.stdout: mon osd allow pg remap = true 2023-01-24T10:44:50.120 INFO:teuthology.orchestra.run.smithi182.stdout: mon warn on legacy crush tunables = false 2023-01-24T10:44:50.120 INFO:teuthology.orchestra.run.smithi182.stdout: mon warn on crush straw calc version zero = false 2023-01-24T10:44:50.120 INFO:teuthology.orchestra.run.smithi182.stdout: mon warn on no sortbitwise = false 2023-01-24T10:44:50.120 INFO:teuthology.orchestra.run.smithi182.stdout: mon warn on osd down out interval zero = false 2023-01-24T10:44:50.120 INFO:teuthology.orchestra.run.smithi182.stdout: mon warn on too few osds = false 2023-01-24T10:44:50.121 INFO:teuthology.orchestra.run.smithi182.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T10:44:50.121 INFO:teuthology.orchestra.run.smithi182.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T10:44:50.121 INFO:teuthology.orchestra.run.smithi182.stdout: mon_allow_pool_size_one = true 2023-01-24T10:44:50.121 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.121 INFO:teuthology.orchestra.run.smithi182.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T10:44:50.122 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.122 INFO:teuthology.orchestra.run.smithi182.stdout: osd default data pool replay window = 5 2023-01-24T10:44:50.122 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.122 INFO:teuthology.orchestra.run.smithi182.stdout: mon allow pool delete = true 2023-01-24T10:44:50.122 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.122 INFO:teuthology.orchestra.run.smithi182.stdout: mon cluster log file level = debug 2023-01-24T10:44:50.123 INFO:teuthology.orchestra.run.smithi182.stdout: debug asserts on shutdown = true 2023-01-24T10:44:50.123 INFO:teuthology.orchestra.run.smithi182.stdout: mon health detail to clog = false 2023-01-24T10:44:50.123 INFO:teuthology.orchestra.run.smithi182.stdout: mon host = "172.21.15.78,[v2:172.21.15.78:3301,v1:172.21.15.78:6790],172.21.15.182" 2023-01-24T10:44:50.123 INFO:teuthology.orchestra.run.smithi182.stdout: mon client directed command retry = 5 2023-01-24T10:44:50.123 INFO:teuthology.orchestra.run.smithi182.stdout: mon election default strategy = 1 2023-01-24T10:44:50.124 INFO:teuthology.orchestra.run.smithi182.stdout: ms bind msgr1 = False 2023-01-24T10:44:50.124 INFO:teuthology.orchestra.run.smithi182.stdout: ms bind msgr2 = True 2023-01-24T10:44:50.124 INFO:teuthology.orchestra.run.smithi182.stdout: ms inject socket failures = 5000 2023-01-24T10:44:50.124 INFO:teuthology.orchestra.run.smithi182.stdout: ms type = async 2023-01-24T10:44:50.124 INFO:teuthology.orchestra.run.smithi182.stdout: fsid = 74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12 2023-01-24T10:44:50.125 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.125 INFO:teuthology.orchestra.run.smithi182.stdout:[osd] 2023-01-24T10:44:50.125 INFO:teuthology.orchestra.run.smithi182.stdout: osd journal size = 100 2023-01-24T10:44:50.125 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.125 INFO:teuthology.orchestra.run.smithi182.stdout: osd scrub load threshold = 5.0 2023-01-24T10:44:50.126 INFO:teuthology.orchestra.run.smithi182.stdout: osd scrub max interval = 600 2023-01-24T10:44:50.126 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.126 INFO:teuthology.orchestra.run.smithi182.stdout: osd recover clone overlap = true 2023-01-24T10:44:50.126 INFO:teuthology.orchestra.run.smithi182.stdout: osd recovery max chunk = 1048576 2023-01-24T10:44:50.126 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.126 INFO:teuthology.orchestra.run.smithi182.stdout: osd debug shutdown = true 2023-01-24T10:44:50.127 INFO:teuthology.orchestra.run.smithi182.stdout: osd debug op order = true 2023-01-24T10:44:50.127 INFO:teuthology.orchestra.run.smithi182.stdout: osd debug verify stray on activate = true 2023-01-24T10:44:50.127 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.127 INFO:teuthology.orchestra.run.smithi182.stdout: osd open classes on start = true 2023-01-24T10:44:50.127 INFO:teuthology.orchestra.run.smithi182.stdout: osd debug pg log writeout = true 2023-01-24T10:44:50.128 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.128 INFO:teuthology.orchestra.run.smithi182.stdout: osd deep scrub update digest min age = 30 2023-01-24T10:44:50.128 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.128 INFO:teuthology.orchestra.run.smithi182.stdout: osd map max advance = 10 2023-01-24T10:44:50.128 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.129 INFO:teuthology.orchestra.run.smithi182.stdout: journal zero on create = true 2023-01-24T10:44:50.129 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.129 INFO:teuthology.orchestra.run.smithi182.stdout: filestore ondisk finisher threads = 3 2023-01-24T10:44:50.129 INFO:teuthology.orchestra.run.smithi182.stdout: filestore apply finisher threads = 3 2023-01-24T10:44:50.129 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.129 INFO:teuthology.orchestra.run.smithi182.stdout: bdev debug aio = true 2023-01-24T10:44:50.130 INFO:teuthology.orchestra.run.smithi182.stdout: osd debug misdirected ops = true 2023-01-24T10:44:50.130 INFO:teuthology.orchestra.run.smithi182.stdout: bluestore block size = 96636764160 2023-01-24T10:44:50.130 INFO:teuthology.orchestra.run.smithi182.stdout: bluestore compression algorithm = snappy 2023-01-24T10:44:50.130 INFO:teuthology.orchestra.run.smithi182.stdout: bluestore compression mode = aggressive 2023-01-24T10:44:50.130 INFO:teuthology.orchestra.run.smithi182.stdout: bluestore fsck on mount = True 2023-01-24T10:44:50.131 INFO:teuthology.orchestra.run.smithi182.stdout: bluestore zero block detection = True 2023-01-24T10:44:50.131 INFO:teuthology.orchestra.run.smithi182.stdout: debug bluefs = 20 2023-01-24T10:44:50.131 INFO:teuthology.orchestra.run.smithi182.stdout: debug bluestore = 20 2023-01-24T10:44:50.131 INFO:teuthology.orchestra.run.smithi182.stdout: debug ms = 1 2023-01-24T10:44:50.131 INFO:teuthology.orchestra.run.smithi182.stdout: debug osd = 20 2023-01-24T10:44:50.132 INFO:teuthology.orchestra.run.smithi182.stdout: debug rocksdb = 10 2023-01-24T10:44:50.132 INFO:teuthology.orchestra.run.smithi182.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T10:44:50.132 INFO:teuthology.orchestra.run.smithi182.stdout: mon osd full ratio = 0.9 2023-01-24T10:44:50.132 INFO:teuthology.orchestra.run.smithi182.stdout: mon osd nearfull ratio = 0.8 2023-01-24T10:44:50.132 INFO:teuthology.orchestra.run.smithi182.stdout: osd debug verify cached snaps = True 2023-01-24T10:44:50.133 INFO:teuthology.orchestra.run.smithi182.stdout: osd debug verify missing on start = True 2023-01-24T10:44:50.133 INFO:teuthology.orchestra.run.smithi182.stdout: osd failsafe full ratio = 0.95 2023-01-24T10:44:50.133 INFO:teuthology.orchestra.run.smithi182.stdout: osd objectstore = bluestore 2023-01-24T10:44:50.133 INFO:teuthology.orchestra.run.smithi182.stdout: osd op queue = debug_random 2023-01-24T10:44:50.133 INFO:teuthology.orchestra.run.smithi182.stdout: osd op queue cut off = debug_random 2023-01-24T10:44:50.133 INFO:teuthology.orchestra.run.smithi182.stdout: osd shutdown pgref assert = True 2023-01-24T10:44:50.134 INFO:teuthology.orchestra.run.smithi182.stdout: osd_class_default_list = * 2023-01-24T10:44:50.134 INFO:teuthology.orchestra.run.smithi182.stdout: osd_class_load_list = * 2023-01-24T10:44:50.134 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.134 INFO:teuthology.orchestra.run.smithi182.stdout:[mgr] 2023-01-24T10:44:50.134 INFO:teuthology.orchestra.run.smithi182.stdout: debug ms = 1 2023-01-24T10:44:50.135 INFO:teuthology.orchestra.run.smithi182.stdout: debug mgr = 20 2023-01-24T10:44:50.135 INFO:teuthology.orchestra.run.smithi182.stdout: debug mon = 20 2023-01-24T10:44:50.135 INFO:teuthology.orchestra.run.smithi182.stdout: debug auth = 20 2023-01-24T10:44:50.135 INFO:teuthology.orchestra.run.smithi182.stdout: mon reweight min pgs per osd = 4 2023-01-24T10:44:50.135 INFO:teuthology.orchestra.run.smithi182.stdout: mon reweight min bytes per osd = 10 2023-01-24T10:44:50.136 INFO:teuthology.orchestra.run.smithi182.stdout: mgr/telemetry/nag = false 2023-01-24T10:44:50.136 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.136 INFO:teuthology.orchestra.run.smithi182.stdout:[mon] 2023-01-24T10:44:50.136 INFO:teuthology.orchestra.run.smithi182.stdout: debug ms = 1 2023-01-24T10:44:50.136 INFO:teuthology.orchestra.run.smithi182.stdout: debug mon = 20 2023-01-24T10:44:50.136 INFO:teuthology.orchestra.run.smithi182.stdout: debug paxos = 20 2023-01-24T10:44:50.137 INFO:teuthology.orchestra.run.smithi182.stdout: debug auth = 20 2023-01-24T10:44:50.137 INFO:teuthology.orchestra.run.smithi182.stdout: mon data avail warn = 5 2023-01-24T10:44:50.137 INFO:teuthology.orchestra.run.smithi182.stdout: mon mgr mkfs grace = 240 2023-01-24T10:44:50.137 INFO:teuthology.orchestra.run.smithi182.stdout: mon reweight min pgs per osd = 4 2023-01-24T10:44:50.137 INFO:teuthology.orchestra.run.smithi182.stdout: mon osd reporter subtree level = osd 2023-01-24T10:44:50.138 INFO:teuthology.orchestra.run.smithi182.stdout: mon osd prime pg temp = true 2023-01-24T10:44:50.138 INFO:teuthology.orchestra.run.smithi182.stdout: mon reweight min bytes per osd = 10 2023-01-24T10:44:50.138 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.138 INFO:teuthology.orchestra.run.smithi182.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T10:44:50.138 INFO:teuthology.orchestra.run.smithi182.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T10:44:50.139 INFO:teuthology.orchestra.run.smithi182.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T10:44:50.139 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.139 INFO:teuthology.orchestra.run.smithi182.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T10:44:50.139 INFO:teuthology.orchestra.run.smithi182.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T10:44:50.139 INFO:teuthology.orchestra.run.smithi182.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T10:44:50.139 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.140 INFO:teuthology.orchestra.run.smithi182.stdout: # 1m isn't quite enough 2023-01-24T10:44:50.140 INFO:teuthology.orchestra.run.smithi182.stdout: mon_down_mkfs_grace = 2m 2023-01-24T10:44:50.140 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.140 INFO:teuthology.orchestra.run.smithi182.stdout: mon_warn_on_filestore_osds = false 2023-01-24T10:44:50.141 INFO:teuthology.orchestra.run.smithi182.stdout: mon min osdmap epochs = 50 2023-01-24T10:44:50.141 INFO:teuthology.orchestra.run.smithi182.stdout: mon osdmap full prune interval = 2 2023-01-24T10:44:50.141 INFO:teuthology.orchestra.run.smithi182.stdout: mon osdmap full prune min = 15 2023-01-24T10:44:50.141 INFO:teuthology.orchestra.run.smithi182.stdout: mon osdmap full prune txsize = 2 2023-01-24T10:44:50.141 INFO:teuthology.orchestra.run.smithi182.stdout: mon scrub interval = 300 2023-01-24T10:44:50.141 INFO:teuthology.orchestra.run.smithi182.stdout: paxos service trim min = 10 2023-01-24T10:44:50.142 INFO:teuthology.orchestra.run.smithi182.stdout: 2023-01-24T10:44:50.142 INFO:teuthology.orchestra.run.smithi182.stdout:[client] 2023-01-24T10:44:50.142 INFO:teuthology.orchestra.run.smithi182.stdout: rgw cache enabled = true 2023-01-24T10:44:50.142 INFO:teuthology.orchestra.run.smithi182.stdout: rgw enable ops log = true 2023-01-24T10:44:50.142 INFO:teuthology.orchestra.run.smithi182.stdout: rgw enable usage log = true 2023-01-24T10:44:50.143 INFO:teuthology.orchestra.run.smithi182.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T10:44:50.143 INFO:teuthology.orchestra.run.smithi182.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T10:44:50.143 INFO:teuthology.orchestra.run.smithi182.stdout:[mon.a] 2023-01-24T10:44:50.143 INFO:teuthology.orchestra.run.smithi182.stdout:[mon.c] 2023-01-24T10:44:50.143 INFO:teuthology.orchestra.run.smithi182.stdout:[mon.b] 2023-01-24T10:44:50.144 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T10:44:50.144 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T10:44:50.212 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T10:44:50.212 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:50.213 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T10:44:50.230 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:50.230 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T10:44:50.286 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi078.front.sepia.ceph.com 2023-01-24T10:44:50.287 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:50.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T10:44:50.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T10:44:50.363 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:44:50.363 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:44:50.419 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi182.front.sepia.ceph.com 2023-01-24T10:44:50.419 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:44:50.419 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T10:44:50.420 DEBUG:teuthology.orchestra.run.smithi182:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T10:44:50.457 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:44:50.457 DEBUG:teuthology.orchestra.run.smithi182:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:44:50.514 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T10:44:50.515 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T10:44:50.515 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T10:44:50.567 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T10:44:50.570 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T10:44:50.637 INFO:teuthology.orchestra.run.smithi182.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T10:44:50.639 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T10:44:50.640 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T10:44:50.640 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T10:44:50.683 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T10:44:50.696 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-24T10:44:50.739 INFO:teuthology.orchestra.run.smithi182.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-24T10:44:50.751 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T10:44:50.752 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi182.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T10:44:50.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T10:44:50.778 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T10:44:50.779 INFO:tasks.ceph:role: osd.0 2023-01-24T10:44:50.779 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com 2023-01-24T10:44:50.779 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T10:44:50.852 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:44:50.852 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:44:50.853 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:44:50.853 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:44:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:44:50.853 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:44:50.853 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:44:50.854 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:44:50.854 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:44:50.854 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:44:51.280 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2023-01-24T10:44:51.280 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-01-24T10:44:51.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T10:44:51.469 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T10:44:51.503 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T10:44:51.571 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T10:44:51.571 INFO:tasks.ceph:role: osd.1 2023-01-24T10:44:51.572 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com 2023-01-24T10:44:51.572 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T10:44:51.644 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:44:51.645 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:44:51.645 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:44:51.645 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:44:51.646 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:44:51.646 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:44:51.646 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:44:51.646 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:44:51.646 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:44:51.647 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:44:52.078 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2023-01-24T10:44:52.079 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-01-24T10:44:52.080 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T10:44:52.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T10:44:52.214 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T10:44:52.281 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T10:44:52.281 INFO:tasks.ceph:role: osd.2 2023-01-24T10:44:52.281 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com 2023-01-24T10:44:52.282 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T10:44:52.355 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:44:52.356 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:44:52.356 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:44:52.356 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:44:52.356 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:44:52.357 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:44:52.357 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:44:52.357 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:44:52.357 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:44:52.357 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:44:52.798 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2023-01-24T10:44:52.800 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-01-24T10:44:52.800 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T10:44:52.964 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T10:44:52.997 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-24T10:44:53.065 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T10:44:53.066 INFO:tasks.ceph:role: osd.3 2023-01-24T10:44:53.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com 2023-01-24T10:44:53.066 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T10:44:53.139 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:44:53.140 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:44:53.140 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:44:53.141 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:44:53.141 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:44:53.142 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:44:53.142 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:44:53.142 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:44:53.142 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:44:53.142 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:44:53.591 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2023-01-24T10:44:53.593 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2023-01-24T10:44:53.593 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-24T10:44:53.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-24T10:44:53.802 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:44:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:53.941+0000 7f318081c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-24T10:44:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:53.942+0000 7f318081c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T10:44:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:53.942+0000 7f318081c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T10:44:53.945 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:53.942+0000 7f318081c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T10:44:53.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:53.942+0000 7f318081c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T10:44:53.946 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:53.942+0000 7f318081c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-24T10:44:54.476 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:54.474+0000 7f318081c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:44:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:55.239+0000 7f318081c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:44:56.009 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:56.006+0000 7f318081c5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:44:56.815 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T10:44:56.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:44:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:56.913+0000 7fa0680705c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-24T10:44:56.916 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:56.913+0000 7fa0680705c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T10:44:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:56.913+0000 7fa0680705c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T10:44:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:56.913+0000 7fa0680705c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T10:44:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:56.913+0000 7fa0680705c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T10:44:56.918 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:56.914+0000 7fa0680705c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-24T10:44:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:57.444+0000 7fa0680705c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:44:58.211 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:58.208+0000 7fa0680705c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:44:58.975 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:58.972+0000 7fa0680705c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:44:59.774 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T10:44:59.806 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:44:59.902 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:59.899+0000 7fbf10f4b5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-24T10:44:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:59.899+0000 7fbf10f4b5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T10:44:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:59.899+0000 7fbf10f4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T10:44:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:59.900+0000 7fbf10f4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T10:44:59.903 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:59.900+0000 7fbf10f4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T10:44:59.904 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:44:59.900+0000 7fbf10f4b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-24T10:45:00.435 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:00.432+0000 7fbf10f4b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:01.199 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:01.196+0000 7fbf10f4b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:01.962 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:01.959+0000 7fbf10f4b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:02.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T10:45:02.793 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:45:02.886 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:02.882+0000 7fe3fb9a65c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-24T10:45:02.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:02.882+0000 7fe3fb9a65c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-24T10:45:02.887 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:02.883+0000 7fe3fb9a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T10:45:02.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:02.883+0000 7fe3fb9a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T10:45:02.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:02.883+0000 7fe3fb9a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T10:45:02.888 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:02.883+0000 7fe3fb9a65c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-24T10:45:03.414 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:03.411+0000 7fe3fb9a65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:04.184 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:04.181+0000 7fe3fb9a65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:04.949 INFO:teuthology.orchestra.run.smithi078.stderr:2023-01-24T10:45:04.946+0000 7fe3fb9a65c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:05.750 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-24T10:45:05.783 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-24T10:45:05.816 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T10:45:05.816 INFO:tasks.ceph:role: osd.4 2023-01-24T10:45:05.817 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi182.front.sepia.ceph.com 2023-01-24T10:45:05.817 DEBUG:teuthology.orchestra.run.smithi182:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T10:45:05.898 INFO:teuthology.orchestra.run.smithi182.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:45:05.898 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:45:05.899 INFO:teuthology.orchestra.run.smithi182.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:45:05.899 INFO:teuthology.orchestra.run.smithi182.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:45:05.899 INFO:teuthology.orchestra.run.smithi182.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:45:05.899 INFO:teuthology.orchestra.run.smithi182.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:45:05.899 INFO:teuthology.orchestra.run.smithi182.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:45:05.900 INFO:teuthology.orchestra.run.smithi182.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:45:05.900 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:45:05.900 INFO:teuthology.orchestra.run.smithi182.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:45:06.333 INFO:teuthology.orchestra.run.smithi182.stdout:Discarding blocks...Done. 2023-01-24T10:45:06.335 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi182.front.sepia.ceph.com -o noatime 2023-01-24T10:45:06.336 DEBUG:teuthology.orchestra.run.smithi182:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-24T10:45:06.530 DEBUG:teuthology.orchestra.run.smithi182:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-24T10:45:06.564 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-24T10:45:06.632 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T10:45:06.632 INFO:tasks.ceph:role: osd.5 2023-01-24T10:45:06.633 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi182.front.sepia.ceph.com 2023-01-24T10:45:06.633 DEBUG:teuthology.orchestra.run.smithi182:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T10:45:06.706 INFO:teuthology.orchestra.run.smithi182.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:45:06.707 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:45:06.707 INFO:teuthology.orchestra.run.smithi182.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:45:06.707 INFO:teuthology.orchestra.run.smithi182.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:45:06.707 INFO:teuthology.orchestra.run.smithi182.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:45:06.708 INFO:teuthology.orchestra.run.smithi182.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:45:06.708 INFO:teuthology.orchestra.run.smithi182.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:45:06.708 INFO:teuthology.orchestra.run.smithi182.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:45:06.708 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:45:06.708 INFO:teuthology.orchestra.run.smithi182.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:45:07.123 INFO:teuthology.orchestra.run.smithi182.stdout:Discarding blocks...Done. 2023-01-24T10:45:07.125 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi182.front.sepia.ceph.com -o noatime 2023-01-24T10:45:07.125 DEBUG:teuthology.orchestra.run.smithi182:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-24T10:45:07.230 DEBUG:teuthology.orchestra.run.smithi182:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-24T10:45:07.264 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-24T10:45:07.332 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T10:45:07.332 INFO:tasks.ceph:role: osd.6 2023-01-24T10:45:07.333 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi182.front.sepia.ceph.com 2023-01-24T10:45:07.333 DEBUG:teuthology.orchestra.run.smithi182:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T10:45:07.406 INFO:teuthology.orchestra.run.smithi182.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:45:07.407 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:45:07.407 INFO:teuthology.orchestra.run.smithi182.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:45:07.407 INFO:teuthology.orchestra.run.smithi182.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:45:07.407 INFO:teuthology.orchestra.run.smithi182.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:45:07.407 INFO:teuthology.orchestra.run.smithi182.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:45:07.408 INFO:teuthology.orchestra.run.smithi182.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:45:07.408 INFO:teuthology.orchestra.run.smithi182.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:45:07.408 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:45:07.408 INFO:teuthology.orchestra.run.smithi182.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:45:07.862 INFO:teuthology.orchestra.run.smithi182.stdout:Discarding blocks...Done. 2023-01-24T10:45:07.864 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi182.front.sepia.ceph.com -o noatime 2023-01-24T10:45:07.867 DEBUG:teuthology.orchestra.run.smithi182:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-24T10:45:08.021 DEBUG:teuthology.orchestra.run.smithi182:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-24T10:45:08.055 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-24T10:45:08.123 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T10:45:08.124 INFO:tasks.ceph:role: osd.7 2023-01-24T10:45:08.124 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi182.front.sepia.ceph.com 2023-01-24T10:45:08.125 DEBUG:teuthology.orchestra.run.smithi182:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T10:45:08.199 INFO:teuthology.orchestra.run.smithi182.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T10:45:08.200 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T10:45:08.200 INFO:teuthology.orchestra.run.smithi182.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T10:45:08.200 INFO:teuthology.orchestra.run.smithi182.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T10:45:08.200 INFO:teuthology.orchestra.run.smithi182.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T10:45:08.200 INFO:teuthology.orchestra.run.smithi182.stdout: = sunit=0 swidth=0 blks 2023-01-24T10:45:08.201 INFO:teuthology.orchestra.run.smithi182.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T10:45:08.201 INFO:teuthology.orchestra.run.smithi182.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T10:45:08.201 INFO:teuthology.orchestra.run.smithi182.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T10:45:08.201 INFO:teuthology.orchestra.run.smithi182.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T10:45:08.638 INFO:teuthology.orchestra.run.smithi182.stdout:Discarding blocks...Done. 2023-01-24T10:45:08.640 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi182.front.sepia.ceph.com -o noatime 2023-01-24T10:45:08.640 DEBUG:teuthology.orchestra.run.smithi182:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-24T10:45:08.802 DEBUG:teuthology.orchestra.run.smithi182:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-24T10:45:08.836 DEBUG:teuthology.orchestra.run.smithi182:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:45:08.947 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:08.944+0000 7f7374b6f5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-24T10:45:08.949 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:08.946+0000 7f7374b6f5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-24T10:45:08.949 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:08.946+0000 7f7374b6f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T10:45:08.949 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:08.946+0000 7f7374b6f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T10:45:08.949 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:08.946+0000 7f7374b6f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T10:45:08.950 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:08.946+0000 7f7374b6f5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-24T10:45:09.480 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:09.477+0000 7f7374b6f5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:10.248 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:10.245+0000 7f7374b6f5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:11.020 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:11.017+0000 7f7374b6f5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:11.818 DEBUG:teuthology.orchestra.run.smithi182:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-24T10:45:11.859 DEBUG:teuthology.orchestra.run.smithi182:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:45:11.952 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:11.948+0000 7fec667565c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-24T10:45:11.952 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:11.949+0000 7fec667565c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-24T10:45:11.952 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:11.949+0000 7fec667565c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T10:45:11.953 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:11.949+0000 7fec667565c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T10:45:11.953 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:11.949+0000 7fec667565c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T10:45:11.953 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:11.949+0000 7fec667565c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-24T10:45:12.480 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:12.477+0000 7fec667565c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:13.244 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:13.241+0000 7fec667565c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:14.008 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:14.005+0000 7fec667565c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:14.806 DEBUG:teuthology.orchestra.run.smithi182:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-24T10:45:14.838 DEBUG:teuthology.orchestra.run.smithi182:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:45:14.930 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:14.927+0000 7fc9a042e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-24T10:45:14.931 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:14.927+0000 7fc9a042e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-24T10:45:14.931 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:14.927+0000 7fc9a042e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T10:45:14.931 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:14.927+0000 7fc9a042e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T10:45:14.931 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:14.927+0000 7fc9a042e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T10:45:14.932 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:14.928+0000 7fc9a042e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-24T10:45:15.468 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:15.465+0000 7fc9a042e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:16.233 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:16.230+0000 7fc9a042e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:16.998 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:16.995+0000 7fc9a042e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:17.797 DEBUG:teuthology.orchestra.run.smithi182:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-24T10:45:17.829 DEBUG:teuthology.orchestra.run.smithi182:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:45:17.923 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:17.920+0000 7fc607ac05c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-24T10:45:17.924 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:17.920+0000 7fc607ac05c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-24T10:45:17.924 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:17.920+0000 7fc607ac05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T10:45:17.924 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:17.920+0000 7fc607ac05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T10:45:17.924 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:17.920+0000 7fc607ac05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T10:45:17.925 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:17.920+0000 7fc607ac05c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-24T10:45:18.450 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:18.447+0000 7fc607ac05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:19.218 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:19.215+0000 7fc607ac05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:19.983 INFO:teuthology.orchestra.run.smithi182.stderr:2023-01-24T10:45:19.980+0000 7fc607ac05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-24T10:45:20.787 DEBUG:teuthology.orchestra.run.smithi182:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-24T10:45:20.818 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T10:45:20.819 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:20.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-24T10:45:20.853 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:20.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T10:45:20.923 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:20.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T10:45:20.992 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:20.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T10:45:21.059 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:21.060 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-24T10:45:21.126 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:21.127 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T10:45:21.159 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:21.159 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-24T10:45:21.231 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:21.231 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-24T10:45:21.299 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:21.300 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-24T10:45:21.368 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:21.369 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-24T10:45:21.436 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:21.436 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T10:45:21.454 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:21.454 DEBUG:teuthology.orchestra.run.smithi182:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-24T10:45:21.493 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T10:45:21.493 DEBUG:teuthology.orchestra.run.smithi078:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T10:45:21.496 DEBUG:teuthology.orchestra.run.smithi182:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T10:45:21.536 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.y] 2023-01-24T10:45:21.536 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCits9j2dvBIRAAw9lYsu62iYrAN4dC5HEMFQ== 2023-01-24T10:45:21.536 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.0] 2023-01-24T10:45:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQClts9jrkU1OBAAKk+fO/Akjn3mp7bdbjBFiQ== 2023-01-24T10:45:21.537 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.1] 2023-01-24T10:45:21.537 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCots9jNTqANhAAoGdfvfQq3qYNIv2l9KG0og== 2023-01-24T10:45:21.537 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.2] 2023-01-24T10:45:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCrts9jK4ayNRAAmIxPvSlX+SBN5GfJbqkcWw== 2023-01-24T10:45:21.538 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.3] 2023-01-24T10:45:21.538 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCuts9j+jOvNBAATYEWidb/9Ohpmf4jGtpYvw== 2023-01-24T10:45:21.538 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.x] 2023-01-24T10:45:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCits9ju5feJRAALPi/G8yDgbUC3+jEVjnJ+g== 2023-01-24T10:45:21.539 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.4] 2023-01-24T10:45:21.539 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQC0ts9jPydkOBAAUNSzunMKRAg4jMElA7Lgyw== 2023-01-24T10:45:21.540 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.5] 2023-01-24T10:45:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQC3ts9jRGugOBAAiFVx4DNGiKgjt+/Api7haw== 2023-01-24T10:45:21.540 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.6] 2023-01-24T10:45:21.540 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQC6ts9jDP5bNxAAq70jWKg+pHN8tHeE/tN04g== 2023-01-24T10:45:21.541 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.7] 2023-01-24T10:45:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQC9ts9jyGbqNhAA+OaHUDdX2cFCwgDAVQTewg== 2023-01-24T10:45:21.541 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2023-01-24T10:45:21.541 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCits9jpXmmKBAAWBtN8ZLI5Ap2T7XiCYuXiQ== 2023-01-24T10:45:21.541 INFO:teuthology.orchestra.run.smithi078.stdout:[client.1] 2023-01-24T10:45:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCits9jZbTzKxAAPjKRRiV7LOr4OMkazVdAbA== 2023-01-24T10:45:21.558 INFO:teuthology.orchestra.run.smithi182.stdout:[mgr.y] 2023-01-24T10:45:21.558 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQCits9j2dvBIRAAw9lYsu62iYrAN4dC5HEMFQ== 2023-01-24T10:45:21.558 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.0] 2023-01-24T10:45:21.559 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQClts9jrkU1OBAAKk+fO/Akjn3mp7bdbjBFiQ== 2023-01-24T10:45:21.559 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.1] 2023-01-24T10:45:21.559 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQCots9jNTqANhAAoGdfvfQq3qYNIv2l9KG0og== 2023-01-24T10:45:21.559 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.2] 2023-01-24T10:45:21.560 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQCrts9jK4ayNRAAmIxPvSlX+SBN5GfJbqkcWw== 2023-01-24T10:45:21.560 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.3] 2023-01-24T10:45:21.560 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQCuts9j+jOvNBAATYEWidb/9Ohpmf4jGtpYvw== 2023-01-24T10:45:21.560 INFO:teuthology.orchestra.run.smithi182.stdout:[mgr.x] 2023-01-24T10:45:21.560 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQCits9ju5feJRAALPi/G8yDgbUC3+jEVjnJ+g== 2023-01-24T10:45:21.561 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.4] 2023-01-24T10:45:21.561 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQC0ts9jPydkOBAAUNSzunMKRAg4jMElA7Lgyw== 2023-01-24T10:45:21.561 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.5] 2023-01-24T10:45:21.561 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQC3ts9jRGugOBAAiFVx4DNGiKgjt+/Api7haw== 2023-01-24T10:45:21.562 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.6] 2023-01-24T10:45:21.562 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQC6ts9jDP5bNxAAq70jWKg+pHN8tHeE/tN04g== 2023-01-24T10:45:21.562 INFO:teuthology.orchestra.run.smithi182.stdout:[osd.7] 2023-01-24T10:45:21.562 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQC9ts9jyGbqNhAA+OaHUDdX2cFCwgDAVQTewg== 2023-01-24T10:45:21.562 INFO:teuthology.orchestra.run.smithi182.stdout:[client.0] 2023-01-24T10:45:21.563 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQCits9jpXmmKBAAWBtN8ZLI5Ap2T7XiCYuXiQ== 2023-01-24T10:45:21.563 INFO:teuthology.orchestra.run.smithi182.stdout:[client.1] 2023-01-24T10:45:21.563 INFO:teuthology.orchestra.run.smithi182.stdout: key = AQCits9jZbTzKxAAPjKRRiV7LOr4OMkazVdAbA== 2023-01-24T10:45:21.564 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T10:45:21.585 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T10:45:21.658 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.661 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.717 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.769 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.771 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.823 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.829 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.881 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T10:45:21.883 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T10:45:21.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.938 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.990 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:21.993 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:22.045 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:22.047 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:22.100 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:22.102 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T10:45:22.155 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T10:45:22.158 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T10:45:22.213 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T10:45:22.216 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T10:45:22.271 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T10:45:22.272 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T10:45:22.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T10:45:22.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T10:45:22.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-24T10:45:22.489 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T10:45:22.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-24T10:45:22.612 DEBUG:teuthology.orchestra.run.smithi182:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-24T10:45:22.639 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T10:45:22.737 DEBUG:teuthology.orchestra.run.smithi182:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-24T10:45:22.764 DEBUG:teuthology.orchestra.run.smithi078:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:45:22.766 DEBUG:teuthology.orchestra.run.smithi182:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T10:45:22.820 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T10:45:22.820 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T10:45:22.821 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T10:45:22.823 INFO:tasks.ceph.mon.a:Started 2023-01-24T10:45:22.823 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-24T10:45:22.824 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-24T10:45:22.826 INFO:tasks.ceph.mon.c:Started 2023-01-24T10:45:22.826 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-24T10:45:22.826 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-24T10:45:22.862 INFO:tasks.ceph.mon.b:Started 2023-01-24T10:45:22.862 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T10:45:22.863 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-24T10:45:22.863 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-24T10:45:22.865 INFO:tasks.ceph.mgr.y:Started 2023-01-24T10:45:22.866 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T10:45:22.866 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T10:45:22.868 INFO:tasks.ceph.mgr.x:Started 2023-01-24T10:45:22.868 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T10:45:22.869 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T10:45:28.296 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2023-01-24T10:45:28.308 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T10:45:28.308 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:28.309 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T10:45:28.327 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:28.325+0000 7f9c72b90200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T10:45:28.335 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:28.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T10:45:28.395 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:28.393+0000 7f9c72b90200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T10:45:28.405 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:28.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T10:45:28.470 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:45:28.471 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-24T10:45:28.474 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:28.472+0000 7f9c72b90200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T10:45:28.500 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:28.500 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-24T10:45:28.532 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:28.533 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-24T10:45:28.555 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:28.553+0000 7f9c72b90200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T10:45:28.601 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:28.602 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-24T10:45:28.670 DEBUG:teuthology.orchestra.run.smithi182:> set -ex 2023-01-24T10:45:28.671 DEBUG:teuthology.orchestra.run.smithi182:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-24T10:45:28.692 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:28.690+0000 7f9c72b90200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T10:45:28.738 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new 23bee10f-7424-40dd-87ce-2bff7d2da2e8 0 2023-01-24T10:45:28.993 INFO:teuthology.orchestra.run.smithi182.stdout:0 2023-01-24T10:45:29.006 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new 9f54a08d-65f7-4b99-8947-a8f52f2870f0 1 2023-01-24T10:45:29.021 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:29.020+0000 7f9c72b90200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-24T10:45:29.058 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.055+0000 7f00769a4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T10:45:29.094 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:29.092+0000 7f9c72b90200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T10:45:29.123 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.120+0000 7f00769a4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T10:45:29.200 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.197+0000 7f00769a4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T10:45:29.222 INFO:teuthology.orchestra.run.smithi182.stdout:1 2023-01-24T10:45:29.223 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:29.221+0000 7f9c72b90200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T10:45:29.235 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new a2be21c3-093a-45b2-b1ce-58f6cf1bf494 2 2023-01-24T10:45:29.282 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.279+0000 7f00769a4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T10:45:29.311 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:29.309+0000 7f9c72b90200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T10:45:29.427 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.424+0000 7f00769a4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T10:45:29.451 INFO:teuthology.orchestra.run.smithi182.stdout:2 2023-01-24T10:45:29.464 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new db4f3cfd-197f-41cc-b8f0-d06ff4266088 3 2023-01-24T10:45:29.693 INFO:teuthology.orchestra.run.smithi182.stdout:3 2023-01-24T10:45:29.706 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new 3d4e802a-dcc6-4e5e-be80-1ca9e877b39b 4 2023-01-24T10:45:29.734 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:29.732+0000 7f9c72b90200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T10:45:29.785 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.782+0000 7f00769a4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-24T10:45:29.864 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.861+0000 7f00769a4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T10:45:29.921 INFO:teuthology.orchestra.run.smithi182.stdout:4 2023-01-24T10:45:29.933 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new 44b94d54-c01f-4bc9-a730-00f2001425d0 5 2023-01-24T10:45:29.992 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:29.989+0000 7f00769a4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T10:45:30.075 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:30.072+0000 7f00769a4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T10:45:30.148 INFO:teuthology.orchestra.run.smithi182.stdout:5 2023-01-24T10:45:30.161 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new 4bc9cfd0-08e7-42c5-a17d-6af8e6eaf7ca 6 2023-01-24T10:45:30.280 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:30.278+0000 7f9c72b90200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T10:45:30.379 INFO:teuthology.orchestra.run.smithi182.stdout:6 2023-01-24T10:45:30.392 DEBUG:teuthology.orchestra.run.smithi182:> sudo ceph --cluster ceph osd new b3446266-ea53-486a-a95b-4110e741f963 7 2023-01-24T10:45:30.433 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:30.431+0000 7f9c72b90200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T10:45:30.530 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:30.527+0000 7f00769a4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T10:45:30.610 INFO:teuthology.orchestra.run.smithi182.stdout:7 2023-01-24T10:45:30.624 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T10:45:30.624 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T10:45:30.626 INFO:tasks.ceph.osd.0:Started 2023-01-24T10:45:30.627 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T10:45:30.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T10:45:30.629 INFO:tasks.ceph.osd.1:Started 2023-01-24T10:45:30.629 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T10:45:30.630 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T10:45:30.632 INFO:tasks.ceph.osd.2:Started 2023-01-24T10:45:30.632 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-24T10:45:30.632 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-24T10:45:30.635 INFO:tasks.ceph.osd.3:Started 2023-01-24T10:45:30.635 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-24T10:45:30.635 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-24T10:45:30.637 INFO:tasks.ceph.osd.4:Started 2023-01-24T10:45:30.638 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-24T10:45:30.638 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-24T10:45:30.640 INFO:tasks.ceph.osd.5:Started 2023-01-24T10:45:30.641 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-24T10:45:30.641 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-24T10:45:30.643 INFO:tasks.ceph.osd.6:Started 2023-01-24T10:45:30.644 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-24T10:45:30.644 DEBUG:teuthology.orchestra.run.smithi182:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-24T10:45:30.646 INFO:tasks.ceph.osd.7:Started 2023-01-24T10:45:30.647 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T10:45:30.869 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:45:30.869 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"fsid":"74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12","created":"2023-01-24T10:45:28.177830+0000","modified":"2023-01-24T10:45:30.603256+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T10:45:30.603256+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"23bee10f-7424-40dd-87ce-2bff7d2da2e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9f54a08d-65f7-4b99-8947-a8f52f2870f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2be21c3-093a-45b2-b1ce-58f6cf1bf494","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db4f3cfd-197f-41cc-b8f0-d06ff4266088","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3d4e802a-dcc6-4e5e-be80-1ca9e877b39b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"44b94d54-c01f-4bc9-a730-00f2001425d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4bc9cfd0-08e7-42c5-a17d-6af8e6eaf7ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b3446266-ea53-486a-a95b-4110e741f963","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T10:45:30.882 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T10:45:30.882 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T10:45:31.107 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:31.104+0000 7f00769a4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T10:45:31.157 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:31.156+0000 7f9c72b90200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T10:45:31.233 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:31.231+0000 7f9c72b90200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T10:45:31.258 INFO:tasks.ceph.osd.4.smithi182.stderr:2023-01-24T10:45:31.255+0000 7f899b88b5c0 -1 Falling back to public interface 2023-01-24T10:45:31.258 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-01-24T10:45:31.255+0000 7f591c4095c0 -1 Falling back to public interface 2023-01-24T10:45:31.259 INFO:tasks.ceph.osd.5.smithi182.stderr:2023-01-24T10:45:31.255+0000 7f95c7ea55c0 -1 Falling back to public interface 2023-01-24T10:45:31.259 INFO:tasks.ceph.osd.4.smithi182.stderr:2023-01-24T10:45:31.255+0000 7f8992c49700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-01-24T10:45:31.259 INFO:tasks.ceph.osd.5.smithi182.stderr:2023-01-24T10:45:31.255+0000 7f95bf263700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-01-24T10:45:31.260 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-01-24T10:45:31.258+0000 7f8974aba5c0 -1 Falling back to public interface 2023-01-24T10:45:31.261 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-01-24T10:45:31.259+0000 7efd1e0535c0 -1 Falling back to public interface 2023-01-24T10:45:31.264 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:31.261+0000 7f00769a4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T10:45:31.265 INFO:tasks.ceph.osd.6.smithi182.stderr:2023-01-24T10:45:31.262+0000 7f38ef7bd5c0 -1 Falling back to public interface 2023-01-24T10:45:31.266 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-01-24T10:45:31.264+0000 7fa68683b5c0 -1 Falling back to public interface 2023-01-24T10:45:31.271 INFO:tasks.ceph.osd.7.smithi182.stderr:2023-01-24T10:45:31.268+0000 7f03d023b5c0 -1 Falling back to public interface 2023-01-24T10:45:31.371 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:31.369+0000 7f9c72b90200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T10:45:31.527 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:31.525+0000 7f9c72b90200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T10:45:31.657 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:31.656+0000 7f9c72b90200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T10:45:31.686 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T10:45:31.881 INFO:teuthology.misc.health.smithi078.stdout: 2023-01-24T10:45:31.881 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":10,"fsid":"74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12","created":"2023-01-24T10:45:28.177830+0000","modified":"2023-01-24T10:45:30.603256+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T10:45:30.603256+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"23bee10f-7424-40dd-87ce-2bff7d2da2e8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9f54a08d-65f7-4b99-8947-a8f52f2870f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a2be21c3-093a-45b2-b1ce-58f6cf1bf494","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"db4f3cfd-197f-41cc-b8f0-d06ff4266088","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3d4e802a-dcc6-4e5e-be80-1ca9e877b39b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"44b94d54-c01f-4bc9-a730-00f2001425d0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4bc9cfd0-08e7-42c5-a17d-6af8e6eaf7ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b3446266-ea53-486a-a95b-4110e741f963","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T10:45:31.892 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-24T10:45:31.936 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:31.933+0000 7f00769a4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T10:45:32.009 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:32.007+0000 7f9c72b90200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T10:45:32.010 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:32.007+0000 7f00769a4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T10:45:32.147 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:32.144+0000 7f00769a4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T10:45:32.159 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:32.157+0000 7f9c72b90200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T10:45:32.272 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:32.270+0000 7f9c72b90200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T10:45:32.302 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:32.299+0000 7f00769a4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T10:45:32.415 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:32.413+0000 7f9c72b90200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T10:45:32.434 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:32.430+0000 7f00769a4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T10:45:32.511 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:32.509+0000 7f9c72b90200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T10:45:32.780 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:32.777+0000 7f00769a4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T10:45:32.860 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:32.859+0000 7f9c72b90200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T10:45:32.931 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:32.928+0000 7f00769a4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T10:45:33.045 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:33.043+0000 7f00769a4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T10:45:33.189 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:33.186+0000 7f00769a4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T10:45:33.287 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:33.284+0000 7f00769a4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T10:45:33.430 INFO:tasks.ceph.mgr.y.smithi078.stderr:2023-01-24T10:45:33.427+0000 7f9c72b90200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T10:45:33.641 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:33.638+0000 7f00769a4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T10:45:34.286 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:34.283+0000 7f00769a4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T10:45:34.802 INFO:tasks.ceph.osd.4.smithi182.stderr:2023-01-24T10:45:34.800+0000 7f899b88b5c0 -1 osd.4 0 log_to_monitors true 2023-01-24T10:45:34.818 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-01-24T10:45:34.815+0000 7fa68683b5c0 -1 osd.3 0 log_to_monitors true 2023-01-24T10:45:34.818 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-01-24T10:45:34.815+0000 7f8974aba5c0 -1 osd.1 0 log_to_monitors true 2023-01-24T10:45:34.828 INFO:tasks.ceph.osd.5.smithi182.stderr:2023-01-24T10:45:34.827+0000 7f95c7ea55c0 -1 osd.5 0 log_to_monitors true 2023-01-24T10:45:34.832 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-01-24T10:45:34.830+0000 7efd1e0535c0 -1 osd.0 0 log_to_monitors true 2023-01-24T10:45:34.836 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-01-24T10:45:34.834+0000 7f591c4095c0 -1 osd.2 0 log_to_monitors true 2023-01-24T10:45:34.840 INFO:tasks.ceph.osd.7.smithi182.stderr:2023-01-24T10:45:34.838+0000 7f03d023b5c0 -1 osd.7 0 log_to_monitors true 2023-01-24T10:45:34.841 INFO:tasks.ceph.osd.6.smithi182.stderr:2023-01-24T10:45:34.840+0000 7f38ef7bd5c0 -1 osd.6 0 log_to_monitors true 2023-01-24T10:45:35.308 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:35.306+0000 7f002ed55700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-24T10:45:35.309 INFO:tasks.ceph.mgr.x.smithi182.stderr:2023-01-24T10:45:35.307+0000 7f002ed55700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-24T10:45:36.366 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-01-24T10:45:36.363+0000 7efd11c0a700 -1 osd.0 0 waiting for initial osdmap 2023-01-24T10:45:36.366 INFO:tasks.ceph.osd.7.smithi182.stderr:2023-01-24T10:45:36.364+0000 7f03c3df2700 -1 osd.7 0 waiting for initial osdmap 2023-01-24T10:45:36.367 INFO:tasks.ceph.osd.4.smithi182.stderr:2023-01-24T10:45:36.365+0000 7f898f442700 -1 osd.4 0 waiting for initial osdmap 2023-01-24T10:45:36.367 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-01-24T10:45:36.365+0000 7f590ffc0700 -1 osd.2 0 waiting for initial osdmap 2023-01-24T10:45:36.368 INFO:tasks.ceph.osd.6.smithi182.stderr:2023-01-24T10:45:36.366+0000 7f38e3374700 -1 osd.6 0 waiting for initial osdmap 2023-01-24T10:45:36.372 INFO:tasks.ceph.osd.0.smithi078.stderr:2023-01-24T10:45:36.369+0000 7efd0dfed700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:36.374 INFO:tasks.ceph.osd.5.smithi182.stderr:2023-01-24T10:45:36.373+0000 7f95bba5c700 -1 osd.5 0 waiting for initial osdmap 2023-01-24T10:45:36.375 INFO:tasks.ceph.osd.7.smithi182.stderr:2023-01-24T10:45:36.373+0000 7f03c01d5700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:36.375 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-01-24T10:45:36.372+0000 7f8968671700 -1 osd.1 0 waiting for initial osdmap 2023-01-24T10:45:36.376 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-01-24T10:45:36.372+0000 7fa67a3f2700 -1 osd.3 0 waiting for initial osdmap 2023-01-24T10:45:36.378 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-01-24T10:45:36.375+0000 7f590c3a3700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:36.379 INFO:tasks.ceph.osd.5.smithi182.stderr:2023-01-24T10:45:36.377+0000 7f95b7e3f700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:36.379 INFO:tasks.ceph.osd.6.smithi182.stderr:2023-01-24T10:45:36.377+0000 7f38df757700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:36.380 INFO:tasks.ceph.osd.4.smithi182.stderr:2023-01-24T10:45:36.378+0000 7f898b825700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:36.383 INFO:tasks.ceph.osd.3.smithi078.stderr:2023-01-24T10:45:36.381+0000 7fa6767d5700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:36.384 INFO:tasks.ceph.osd.1.smithi078.stderr:2023-01-24T10:45:36.381+0000 7f8964a54700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T10:45:38.699 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T10:45:39.016 INFO:teuthology.misc.health.smithi078.stdout: 2023-01-24T10:45:39.017 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":14,"fsid":"74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12","created":"2023-01-24T10:45:28.177830+0000","modified":"2023-01-24T10:45:38.376247+0000","last_up_change":"2023-01-24T10:45:37.365715+0000","last_in_change":"2023-01-24T10:45:30.603256+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T10:45:38.345681+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"23bee10f-7424-40dd-87ce-2bff7d2da2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":91032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":91032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6811","nonce":91032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":91032}]},"public_addr":"172.21.15.78:6808/91032","cluster_addr":"172.21.15.78:6809/91032","heartbeat_back_addr":"172.21.15.78:6811/91032","heartbeat_front_addr":"172.21.15.78:6810/91032","state":["exists","up"]},{"osd":1,"uuid":"9f54a08d-65f7-4b99-8947-a8f52f2870f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":91033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6805","nonce":91033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6807","nonce":91033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":91033}]},"public_addr":"172.21.15.78:6804/91033","cluster_addr":"172.21.15.78:6805/91033","heartbeat_back_addr":"172.21.15.78:6807/91033","heartbeat_front_addr":"172.21.15.78:6806/91033","state":["exists","up"]},{"osd":2,"uuid":"a2be21c3-093a-45b2-b1ce-58f6cf1bf494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":91031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6801","nonce":91031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6803","nonce":91031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":91031}]},"public_addr":"172.21.15.78:6800/91031","cluster_addr":"172.21.15.78:6801/91031","heartbeat_back_addr":"172.21.15.78:6803/91031","heartbeat_front_addr":"172.21.15.78:6802/91031","state":["exists","up"]},{"osd":3,"uuid":"db4f3cfd-197f-41cc-b8f0-d06ff4266088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":91044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6813","nonce":91044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":91044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":91044}]},"public_addr":"172.21.15.78:6812/91044","cluster_addr":"172.21.15.78:6813/91044","heartbeat_back_addr":"172.21.15.78:6815/91044","heartbeat_front_addr":"172.21.15.78:6814/91044","state":["exists","up"]},{"osd":4,"uuid":"3d4e802a-dcc6-4e5e-be80-1ca9e877b39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6801","nonce":90954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6803","nonce":90954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6807","nonce":90954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6805","nonce":90954}]},"public_addr":"172.21.15.182:6801/90954","cluster_addr":"172.21.15.182:6803/90954","heartbeat_back_addr":"172.21.15.182:6807/90954","heartbeat_front_addr":"172.21.15.182:6805/90954","state":["exists","up"]},{"osd":5,"uuid":"44b94d54-c01f-4bc9-a730-00f2001425d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":90955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":90955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":90955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":90955}]},"public_addr":"172.21.15.182:6800/90955","cluster_addr":"172.21.15.182:6802/90955","heartbeat_back_addr":"172.21.15.182:6806/90955","heartbeat_front_addr":"172.21.15.182:6804/90955","state":["exists","up"]},{"osd":6,"uuid":"4bc9cfd0-08e7-42c5-a17d-6af8e6eaf7ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":90974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6809","nonce":90974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6811","nonce":90974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":90974}]},"public_addr":"172.21.15.182:6808/90974","cluster_addr":"172.21.15.182:6809/90974","heartbeat_back_addr":"172.21.15.182:6811/90974","heartbeat_front_addr":"172.21.15.182:6810/90974","state":["exists","up"]},{"osd":7,"uuid":"b3446266-ea53-486a-a95b-4110e741f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":90985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6813","nonce":90985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6815","nonce":90985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":90985}]},"public_addr":"172.21.15.182:6812/90985","cluster_addr":"172.21.15.182:6813/90985","heartbeat_back_addr":"172.21.15.182:6815/90985","heartbeat_front_addr":"172.21.15.182:6814/90985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.839281+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.813012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.809585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.792101+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.832737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.833881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.797283+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.866955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T10:45:39.029 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-24T10:45:39.029 INFO:tasks.ceph:Creating RBD pool 2023-01-24T10:45:39.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T10:45:39.393 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'rbd' created 2023-01-24T10:45:39.405 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T10:45:40.415 INFO:teuthology.orchestra.run.smithi078.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T10:45:40.428 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T10:45:40.429 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T10:45:40.429 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T10:45:40.430 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T10:45:40.430 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T10:45:40.794 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:45:40.807 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":4,"active_gid":4122,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6816","nonce":90351}]},"active_addr":"172.21.15.182:6816/90351","active_change":"2023-01-24T10:45:34.297401+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4100,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.182:0","nonce":2978290669}]},{"addrvec":[{"type":"v2","addr":"172.21.15.182:0","nonce":2351205728}]},{"addrvec":[{"type":"v2","addr":"172.21.15.182:0","nonce":253854637}]}]}} 2023-01-24T10:45:40.811 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T10:45:40.811 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T10:45:40.812 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T10:45:41.125 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:45:41.126 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"fsid":"74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12","created":"2023-01-24T10:45:28.177830+0000","modified":"2023-01-24T10:45:40.384723+0000","last_up_change":"2023-01-24T10:45:37.365715+0000","last_in_change":"2023-01-24T10:45:30.603256+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T10:45:38.345681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T10:45:39.354348+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"23bee10f-7424-40dd-87ce-2bff7d2da2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":91032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":91032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6811","nonce":91032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":91032}]},"public_addr":"172.21.15.78:6808/91032","cluster_addr":"172.21.15.78:6809/91032","heartbeat_back_addr":"172.21.15.78:6811/91032","heartbeat_front_addr":"172.21.15.78:6810/91032","state":["exists","up"]},{"osd":1,"uuid":"9f54a08d-65f7-4b99-8947-a8f52f2870f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":91033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6805","nonce":91033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6807","nonce":91033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":91033}]},"public_addr":"172.21.15.78:6804/91033","cluster_addr":"172.21.15.78:6805/91033","heartbeat_back_addr":"172.21.15.78:6807/91033","heartbeat_front_addr":"172.21.15.78:6806/91033","state":["exists","up"]},{"osd":2,"uuid":"a2be21c3-093a-45b2-b1ce-58f6cf1bf494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":91031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6801","nonce":91031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6803","nonce":91031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":91031}]},"public_addr":"172.21.15.78:6800/91031","cluster_addr":"172.21.15.78:6801/91031","heartbeat_back_addr":"172.21.15.78:6803/91031","heartbeat_front_addr":"172.21.15.78:6802/91031","state":["exists","up"]},{"osd":3,"uuid":"db4f3cfd-197f-41cc-b8f0-d06ff4266088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":91044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6813","nonce":91044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":91044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":91044}]},"public_addr":"172.21.15.78:6812/91044","cluster_addr":"172.21.15.78:6813/91044","heartbeat_back_addr":"172.21.15.78:6815/91044","heartbeat_front_addr":"172.21.15.78:6814/91044","state":["exists","up"]},{"osd":4,"uuid":"3d4e802a-dcc6-4e5e-be80-1ca9e877b39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6801","nonce":90954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6803","nonce":90954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6807","nonce":90954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6805","nonce":90954}]},"public_addr":"172.21.15.182:6801/90954","cluster_addr":"172.21.15.182:6803/90954","heartbeat_back_addr":"172.21.15.182:6807/90954","heartbeat_front_addr":"172.21.15.182:6805/90954","state":["exists","up"]},{"osd":5,"uuid":"44b94d54-c01f-4bc9-a730-00f2001425d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":90955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":90955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":90955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":90955}]},"public_addr":"172.21.15.182:6800/90955","cluster_addr":"172.21.15.182:6802/90955","heartbeat_back_addr":"172.21.15.182:6806/90955","heartbeat_front_addr":"172.21.15.182:6804/90955","state":["exists","up"]},{"osd":6,"uuid":"4bc9cfd0-08e7-42c5-a17d-6af8e6eaf7ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":90974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6809","nonce":90974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6811","nonce":90974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":90974}]},"public_addr":"172.21.15.182:6808/90974","cluster_addr":"172.21.15.182:6809/90974","heartbeat_back_addr":"172.21.15.182:6811/90974","heartbeat_front_addr":"172.21.15.182:6810/90974","state":["exists","up"]},{"osd":7,"uuid":"b3446266-ea53-486a-a95b-4110e741f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":90985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6813","nonce":90985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6815","nonce":90985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":90985}]},"public_addr":"172.21.15.182:6812/90985","cluster_addr":"172.21.15.182:6813/90985","heartbeat_back_addr":"172.21.15.182:6815/90985","heartbeat_front_addr":"172.21.15.182:6814/90985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.839281+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.813012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.809585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.792101+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.832737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.833881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.797283+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.866955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T10:45:41.140 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T10:45:41.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T10:45:41.452 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:45:41.452 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"fsid":"74ceb6f8-5b04-4b13-b85d-ca0dc5ce3e12","created":"2023-01-24T10:45:28.177830+0000","modified":"2023-01-24T10:45:41.405883+0000","last_up_change":"2023-01-24T10:45:37.365715+0000","last_in_change":"2023-01-24T10:45:30.603256+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T10:45:38.345681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T10:45:39.354348+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"23bee10f-7424-40dd-87ce-2bff7d2da2e8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":91032}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6809","nonce":91032}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6811","nonce":91032}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":91032}]},"public_addr":"172.21.15.78:6808/91032","cluster_addr":"172.21.15.78:6809/91032","heartbeat_back_addr":"172.21.15.78:6811/91032","heartbeat_front_addr":"172.21.15.78:6810/91032","state":["exists","up"]},{"osd":1,"uuid":"9f54a08d-65f7-4b99-8947-a8f52f2870f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":91033}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6805","nonce":91033}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6807","nonce":91033}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":91033}]},"public_addr":"172.21.15.78:6804/91033","cluster_addr":"172.21.15.78:6805/91033","heartbeat_back_addr":"172.21.15.78:6807/91033","heartbeat_front_addr":"172.21.15.78:6806/91033","state":["exists","up"]},{"osd":2,"uuid":"a2be21c3-093a-45b2-b1ce-58f6cf1bf494","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":91031}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6801","nonce":91031}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6803","nonce":91031}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":91031}]},"public_addr":"172.21.15.78:6800/91031","cluster_addr":"172.21.15.78:6801/91031","heartbeat_back_addr":"172.21.15.78:6803/91031","heartbeat_front_addr":"172.21.15.78:6802/91031","state":["exists","up"]},{"osd":3,"uuid":"db4f3cfd-197f-41cc-b8f0-d06ff4266088","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":91044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6813","nonce":91044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6815","nonce":91044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":91044}]},"public_addr":"172.21.15.78:6812/91044","cluster_addr":"172.21.15.78:6813/91044","heartbeat_back_addr":"172.21.15.78:6815/91044","heartbeat_front_addr":"172.21.15.78:6814/91044","state":["exists","up"]},{"osd":4,"uuid":"3d4e802a-dcc6-4e5e-be80-1ca9e877b39b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6801","nonce":90954}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6803","nonce":90954}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6807","nonce":90954}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6805","nonce":90954}]},"public_addr":"172.21.15.182:6801/90954","cluster_addr":"172.21.15.182:6803/90954","heartbeat_back_addr":"172.21.15.182:6807/90954","heartbeat_front_addr":"172.21.15.182:6805/90954","state":["exists","up"]},{"osd":5,"uuid":"44b94d54-c01f-4bc9-a730-00f2001425d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6800","nonce":90955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6802","nonce":90955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6806","nonce":90955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6804","nonce":90955}]},"public_addr":"172.21.15.182:6800/90955","cluster_addr":"172.21.15.182:6802/90955","heartbeat_back_addr":"172.21.15.182:6806/90955","heartbeat_front_addr":"172.21.15.182:6804/90955","state":["exists","up"]},{"osd":6,"uuid":"4bc9cfd0-08e7-42c5-a17d-6af8e6eaf7ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6808","nonce":90974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6809","nonce":90974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6811","nonce":90974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6810","nonce":90974}]},"public_addr":"172.21.15.182:6808/90974","cluster_addr":"172.21.15.182:6809/90974","heartbeat_back_addr":"172.21.15.182:6811/90974","heartbeat_front_addr":"172.21.15.182:6810/90974","state":["exists","up"]},{"osd":7,"uuid":"b3446266-ea53-486a-a95b-4110e741f963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6812","nonce":90985}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6813","nonce":90985}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6815","nonce":90985}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.182:6814","nonce":90985}]},"public_addr":"172.21.15.182:6812/90985","cluster_addr":"172.21.15.182:6813/90985","heartbeat_back_addr":"172.21.15.182:6815/90985","heartbeat_front_addr":"172.21.15.182:6814/90985","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.839281+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.813012+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.809585+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.792101+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.832737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.833881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.797283+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T10:45:35.866955+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T10:45:41.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T10:45:41.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T10:45:41.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T10:45:41.468 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-24T10:45:41.468 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-24T10:45:41.469 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-24T10:45:41.470 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-24T10:45:41.470 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-24T10:45:41.744 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:41.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T10:45:41.745 INFO:teuthology.orchestra.run.smithi078.stdout:55834574850 2023-01-24T10:45:41.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T10:45:41.757 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:41.757 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T10:45:41.768 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:41.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T10:45:41.770 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:41.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T10:45:41.777 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:41.777 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T10:45:41.779 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:41.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T10:45:41.789 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:41.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T10:45:42.365 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:42.366 INFO:teuthology.orchestra.run.smithi078.stdout:55834574849 2023-01-24T10:45:42.369 INFO:teuthology.orchestra.run.smithi078.stdout:55834574850 2023-01-24T10:45:42.384 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:42.385 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:42.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2023-01-24T10:45:42.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.4 2023-01-24T10:45:42.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-24T10:45:42.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.2 2023-01-24T10:45:42.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2023-01-24T10:45:42.420 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:42.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2023-01-24T10:45:42.434 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:42.436 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:42.448 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:42.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2023-01-24T10:45:42.450 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:42.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2023-01-24T10:45:42.463 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:43.391 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T10:45:43.397 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T10:45:43.398 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T10:45:43.406 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T10:45:43.415 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T10:45:43.789 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:43.801 INFO:teuthology.orchestra.run.smithi078.stdout:55834574851 2023-01-24T10:45:43.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2023-01-24T10:45:43.804 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:43.817 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2023-01-24T10:45:43.818 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:43.825 INFO:teuthology.orchestra.run.smithi078.stdout:55834574850 2023-01-24T10:45:43.839 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.4 2023-01-24T10:45:43.839 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:43.931 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:43.940 INFO:teuthology.orchestra.run.smithi078.stdout:55834574852 2023-01-24T10:45:43.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2023-01-24T10:45:43.946 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:43.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.2 2023-01-24T10:45:43.955 DEBUG:teuthology.parallel:result is None 2023-01-24T10:45:43.955 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T10:45:43.955 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T10:45:44.303 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:45:44.305 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-01-24T10:45:44.317 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2023-01-24T10:45:42.302442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":41248,"kb_used_data":1184,"kb_used_omap":0,"kb_used_meta":39936,"kb_avail":754933472,"statfs":{"total":773094113280,"available":773051875328,"internally_reserved":0,"allocated":1212416,"data_stored":1374042,"data_compressed":59094,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":40894464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7],"upper_bound":16},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.408627+0000","last_change":"2023-01-24T10:45:40.412800+0000","last_active":"2023-01-24T10:45:41.408627+0000","last_peered":"2023-01-24T10:45:41.408627+0000","last_clean":"2023-01-24T10:45:41.408627+0000","last_became_active":"2023-01-24T10:45:40.412491+0000","last_became_peered":"2023-01-24T10:45:40.412491+0000","last_unstale":"2023-01-24T10:45:41.408627+0000","last_undegraded":"2023-01-24T10:45:41.408627+0000","last_fullsized":"2023-01-24T10:45:41.408627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T18:40:17.934306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.409084+0000","last_change":"2023-01-24T10:45:40.418448+0000","last_active":"2023-01-24T10:45:41.409084+0000","last_peered":"2023-01-24T10:45:41.409084+0000","last_clean":"2023-01-24T10:45:41.409084+0000","last_became_active":"2023-01-24T10:45:40.418240+0000","last_became_peered":"2023-01-24T10:45:40.418240+0000","last_unstale":"2023-01-24T10:45:41.409084+0000","last_undegraded":"2023-01-24T10:45:41.409084+0000","last_fullsized":"2023-01-24T10:45:41.409084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T13:18:44.467833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:40.420350+0000","last_change":"2023-01-24T10:45:40.415720+0000","last_active":"2023-01-24T10:45:40.420350+0000","last_peered":"2023-01-24T10:45:40.420350+0000","last_clean":"2023-01-24T10:45:40.420350+0000","last_became_active":"2023-01-24T10:45:40.415491+0000","last_became_peered":"2023-01-24T10:45:40.415491+0000","last_unstale":"2023-01-24T10:45:40.420350+0000","last_undegraded":"2023-01-24T10:45:40.420350+0000","last_fullsized":"2023-01-24T10:45:40.420350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T22:43:13.031237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:40.418102+0000","last_change":"2023-01-24T10:45:40.411972+0000","last_active":"2023-01-24T10:45:40.418102+0000","last_peered":"2023-01-24T10:45:40.418102+0000","last_clean":"2023-01-24T10:45:40.418102+0000","last_became_active":"2023-01-24T10:45:40.411782+0000","last_became_peered":"2023-01-24T10:45:40.411782+0000","last_unstale":"2023-01-24T10:45:40.418102+0000","last_undegraded":"2023-01-24T10:45:40.418102+0000","last_fullsized":"2023-01-24T10:45:40.418102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:41:15.309239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:40.428463+0000","last_change":"2023-01-24T10:45:39.391086+0000","last_active":"2023-01-24T10:45:40.428463+0000","last_peered":"2023-01-24T10:45:40.428463+0000","last_clean":"2023-01-24T10:45:40.428463+0000","last_became_active":"2023-01-24T10:45:39.390803+0000","last_became_peered":"2023-01-24T10:45:39.390803+0000","last_unstale":"2023-01-24T10:45:40.428463+0000","last_undegraded":"2023-01-24T10:45:40.428463+0000","last_fullsized":"2023-01-24T10:45:40.428463+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:38.376247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:38.376247+0000","last_clean_scrub_stamp":"2023-01-24T10:45:38.376247+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:39:12.398474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.730768+0000","last_change":"2023-01-24T10:45:40.418018+0000","last_active":"2023-01-24T10:45:41.730768+0000","last_peered":"2023-01-24T10:45:41.730768+0000","last_clean":"2023-01-24T10:45:41.730768+0000","last_became_active":"2023-01-24T10:45:40.417784+0000","last_became_peered":"2023-01-24T10:45:40.417784+0000","last_unstale":"2023-01-24T10:45:41.730768+0000","last_undegraded":"2023-01-24T10:45:41.730768+0000","last_fullsized":"2023-01-24T10:45:41.730768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T15:20:15.917867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:40.411527+0000","last_change":"2023-01-24T10:45:40.409759+0000","last_active":"2023-01-24T10:45:40.411527+0000","last_peered":"2023-01-24T10:45:40.411527+0000","last_clean":"2023-01-24T10:45:40.411527+0000","last_became_active":"2023-01-24T10:45:40.409472+0000","last_became_peered":"2023-01-24T10:45:40.409472+0000","last_unstale":"2023-01-24T10:45:40.411527+0000","last_undegraded":"2023-01-24T10:45:40.411527+0000","last_fullsized":"2023-01-24T10:45:40.411527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:03:18.898609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.730765+0000","last_change":"2023-01-24T10:45:40.415603+0000","last_active":"2023-01-24T10:45:41.730765+0000","last_peered":"2023-01-24T10:45:41.730765+0000","last_clean":"2023-01-24T10:45:41.730765+0000","last_became_active":"2023-01-24T10:45:40.415349+0000","last_became_peered":"2023-01-24T10:45:40.415349+0000","last_unstale":"2023-01-24T10:45:41.730765+0000","last_undegraded":"2023-01-24T10:45:41.730765+0000","last_fullsized":"2023-01-24T10:45:41.730765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T16:40:04.457577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.409072+0000","last_change":"2023-01-24T10:45:40.415684+0000","last_active":"2023-01-24T10:45:41.409072+0000","last_peered":"2023-01-24T10:45:41.409072+0000","last_clean":"2023-01-24T10:45:41.409072+0000","last_became_active":"2023-01-24T10:45:40.415420+0000","last_became_peered":"2023-01-24T10:45:40.415420+0000","last_unstale":"2023-01-24T10:45:41.409072+0000","last_undegraded":"2023-01-24T10:45:41.409072+0000","last_fullsized":"2023-01-24T10:45:41.409072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:41:15.309239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":59094,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5612,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366228,"statfs":{"total":96636764160,"available":96631017472,"internally_reserved":0,"allocated":200704,"data_stored":570294,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5612,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366228,"statfs":{"total":96636764160,"available":96631017472,"internally_reserved":0,"allocated":200704,"data_stored":570294,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":4984,"kb_used_data":112,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366856,"statfs":{"total":96636764160,"available":96631660544,"internally_reserved":0,"allocated":114688,"data_stored":26624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":41366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3479999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T10:45:44.319 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T10:45:44.630 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:45:44.632 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2023-01-24T10:45:44.643 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T10:45:44.302903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":41400,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":754933320,"statfs":{"total":773094113280,"available":773051719680,"internally_reserved":0,"allocated":1302528,"data_stored":1425712,"data_compressed":59094,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000461"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.408627+0000","last_change":"2023-01-24T10:45:40.412800+0000","last_active":"2023-01-24T10:45:41.408627+0000","last_peered":"2023-01-24T10:45:41.408627+0000","last_clean":"2023-01-24T10:45:41.408627+0000","last_became_active":"2023-01-24T10:45:40.412491+0000","last_became_peered":"2023-01-24T10:45:40.412491+0000","last_unstale":"2023-01-24T10:45:41.408627+0000","last_undegraded":"2023-01-24T10:45:41.408627+0000","last_fullsized":"2023-01-24T10:45:41.408627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T18:40:17.934306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.409084+0000","last_change":"2023-01-24T10:45:40.418448+0000","last_active":"2023-01-24T10:45:41.409084+0000","last_peered":"2023-01-24T10:45:41.409084+0000","last_clean":"2023-01-24T10:45:41.409084+0000","last_became_active":"2023-01-24T10:45:40.418240+0000","last_became_peered":"2023-01-24T10:45:40.418240+0000","last_unstale":"2023-01-24T10:45:41.409084+0000","last_undegraded":"2023-01-24T10:45:41.409084+0000","last_fullsized":"2023-01-24T10:45:41.409084+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T13:18:44.467833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.738542+0000","last_change":"2023-01-24T10:45:40.415720+0000","last_active":"2023-01-24T10:45:41.738542+0000","last_peered":"2023-01-24T10:45:41.738542+0000","last_clean":"2023-01-24T10:45:41.738542+0000","last_became_active":"2023-01-24T10:45:40.415491+0000","last_became_peered":"2023-01-24T10:45:40.415491+0000","last_unstale":"2023-01-24T10:45:41.738542+0000","last_undegraded":"2023-01-24T10:45:41.738542+0000","last_fullsized":"2023-01-24T10:45:41.738542+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T22:43:13.031237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.738500+0000","last_change":"2023-01-24T10:45:40.411972+0000","last_active":"2023-01-24T10:45:41.738500+0000","last_peered":"2023-01-24T10:45:41.738500+0000","last_clean":"2023-01-24T10:45:41.738500+0000","last_became_active":"2023-01-24T10:45:40.411782+0000","last_became_peered":"2023-01-24T10:45:40.411782+0000","last_unstale":"2023-01-24T10:45:41.738500+0000","last_undegraded":"2023-01-24T10:45:41.738500+0000","last_fullsized":"2023-01-24T10:45:41.738500+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:41:15.309239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":161,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.738521+0000","last_change":"2023-01-24T10:45:39.391086+0000","last_active":"2023-01-24T10:45:41.738521+0000","last_peered":"2023-01-24T10:45:41.738521+0000","last_clean":"2023-01-24T10:45:41.738521+0000","last_became_active":"2023-01-24T10:45:39.390803+0000","last_became_peered":"2023-01-24T10:45:39.390803+0000","last_unstale":"2023-01-24T10:45:41.738521+0000","last_undegraded":"2023-01-24T10:45:41.738521+0000","last_fullsized":"2023-01-24T10:45:41.738521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:38.376247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:38.376247+0000","last_clean_scrub_stamp":"2023-01-24T10:45:38.376247+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T12:39:12.398474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.730768+0000","last_change":"2023-01-24T10:45:40.418018+0000","last_active":"2023-01-24T10:45:41.730768+0000","last_peered":"2023-01-24T10:45:41.730768+0000","last_clean":"2023-01-24T10:45:41.730768+0000","last_became_active":"2023-01-24T10:45:40.417784+0000","last_became_peered":"2023-01-24T10:45:40.417784+0000","last_unstale":"2023-01-24T10:45:41.730768+0000","last_undegraded":"2023-01-24T10:45:41.730768+0000","last_fullsized":"2023-01-24T10:45:41.730768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T15:20:15.917867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.723395+0000","last_change":"2023-01-24T10:45:40.409759+0000","last_active":"2023-01-24T10:45:41.723395+0000","last_peered":"2023-01-24T10:45:41.723395+0000","last_clean":"2023-01-24T10:45:41.723395+0000","last_became_active":"2023-01-24T10:45:40.409472+0000","last_became_peered":"2023-01-24T10:45:40.409472+0000","last_unstale":"2023-01-24T10:45:41.723395+0000","last_undegraded":"2023-01-24T10:45:41.723395+0000","last_fullsized":"2023-01-24T10:45:41.723395+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:03:18.898609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.730765+0000","last_change":"2023-01-24T10:45:40.415603+0000","last_active":"2023-01-24T10:45:41.730765+0000","last_peered":"2023-01-24T10:45:41.730765+0000","last_clean":"2023-01-24T10:45:41.730765+0000","last_became_active":"2023-01-24T10:45:40.415349+0000","last_became_peered":"2023-01-24T10:45:40.415349+0000","last_unstale":"2023-01-24T10:45:41.730765+0000","last_undegraded":"2023-01-24T10:45:41.730765+0000","last_fullsized":"2023-01-24T10:45:41.730765+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T16:40:04.457577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-01-24T10:45:41.409072+0000","last_change":"2023-01-24T10:45:40.415684+0000","last_active":"2023-01-24T10:45:41.409072+0000","last_peered":"2023-01-24T10:45:41.409072+0000","last_clean":"2023-01-24T10:45:41.409072+0000","last_became_active":"2023-01-24T10:45:40.415420+0000","last_became_peered":"2023-01-24T10:45:40.415420+0000","last_unstale":"2023-01-24T10:45:41.409072+0000","last_undegraded":"2023-01-24T10:45:41.409072+0000","last_fullsized":"2023-01-24T10:45:41.409072+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T10:45:39.383491+0000","last_clean_scrub_stamp":"2023-01-24T10:45:39.383491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:41:15.309239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":1057856,"data_compressed":59094,"data_compressed_allocated":81920,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5588,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366252,"statfs":{"total":96636764160,"available":96631042048,"internally_reserved":0,"allocated":208896,"data_stored":574910,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5588,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366252,"statfs":{"total":96636764160,"available":96631042048,"internally_reserved":0,"allocated":208896,"data_stored":574910,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366760,"statfs":{"total":96636764160,"available":96631562240,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.359}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.927}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3479999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4019999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":528928,"data_compressed":29547,"data_compressed_allocated":40960,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T10:45:44.645 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T10:45:44.645 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T10:45:44.645 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T10:45:44.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T10:45:44.991 INFO:teuthology.orchestra.run.smithi078.stdout: 2023-01-24T10:45:44.991 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T10:45:45.005 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T10:45:45.006 INFO:teuthology.run_tasks:Running task workunit... 2023-01-24T10:45:45.024 INFO:tasks.workunit:Pulling workunits from ref 1a4043421bb637c432ee02d001db3c6ec52c48e0 2023-01-24T10:45:45.025 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-01-24T10:45:45.025 DEBUG:teuthology.orchestra.run.smithi078:> stat -- /home/ubuntu/cephtest/mnt.0 2023-01-24T10:45:45.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T10:45:45.042 INFO:teuthology.orchestra.run.smithi078.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-01-24T10:45:45.043 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-01-24T10:45:45.098 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-01-24T10:45:45.099 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-01-24T10:45:45.154 INFO:tasks.workunit:timeout=3h 2023-01-24T10:45:45.155 INFO:tasks.workunit:cleanup=True 2023-01-24T10:45:45.155 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1a4043421bb637c432ee02d001db3c6ec52c48e0 2023-01-24T10:45:45.212 INFO:tasks.workunit.client.0.smithi078.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-01-24T10:46:32.729 INFO:tasks.workunit.client.0.smithi078.stderr:Note: switching to '1a4043421bb637c432ee02d001db3c6ec52c48e0'. 2023-01-24T10:46:32.730 INFO:tasks.workunit.client.0.smithi078.stderr: 2023-01-24T10:46:32.730 INFO:tasks.workunit.client.0.smithi078.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-01-24T10:46:32.730 INFO:tasks.workunit.client.0.smithi078.stderr:changes and commit them, and you can discard any commits you make in this 2023-01-24T10:46:32.731 INFO:tasks.workunit.client.0.smithi078.stderr:state without impacting any branches by switching back to a branch. 2023-01-24T10:46:32.731 INFO:tasks.workunit.client.0.smithi078.stderr: 2023-01-24T10:46:32.731 INFO:tasks.workunit.client.0.smithi078.stderr:If you want to create a new branch to retain commits you create, you may 2023-01-24T10:46:32.731 INFO:tasks.workunit.client.0.smithi078.stderr:do so (now or later) by using -c with the switch command. Example: 2023-01-24T10:46:32.731 INFO:tasks.workunit.client.0.smithi078.stderr: 2023-01-24T10:46:32.732 INFO:tasks.workunit.client.0.smithi078.stderr: git switch -c 2023-01-24T10:46:32.732 INFO:tasks.workunit.client.0.smithi078.stderr: 2023-01-24T10:46:32.732 INFO:tasks.workunit.client.0.smithi078.stderr:Or undo this operation with: 2023-01-24T10:46:32.732 INFO:tasks.workunit.client.0.smithi078.stderr: 2023-01-24T10:46:32.732 INFO:tasks.workunit.client.0.smithi078.stderr: git switch - 2023-01-24T10:46:32.733 INFO:tasks.workunit.client.0.smithi078.stderr: 2023-01-24T10:46:32.733 INFO:tasks.workunit.client.0.smithi078.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-01-24T10:46:32.733 INFO:tasks.workunit.client.0.smithi078.stderr: 2023-01-24T10:46:32.733 INFO:tasks.workunit.client.0.smithi078.stderr:HEAD is now at 1a4043421bb rgw/fifo: Retry on push to nonexistent part 2023-01-24T10:46:32.734 DEBUG:teuthology.orchestra.run.smithi078:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-01-24T10:46:32.802 INFO:tasks.workunit.client.0.smithi078.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-01-24T10:46:32.804 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-24T10:46:32.804 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-01-24T10:46:33.363 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-01-24T10:46:33.415 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-01-24T10:46:33.452 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-24T10:46:33.454 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-24T10:46:33.454 INFO:tasks.workunit.client.0.smithi078.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-01-24T10:46:33.490 INFO:tasks.workunit.client.0.smithi078.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-24T10:46:33.493 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2023-01-24T10:46:33.493 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-01-24T10:46:33.549 INFO:tasks.workunit:Running workunits matching cls on client.0... 2023-01-24T10:46:33.551 INFO:tasks.workunit:Running workunit cls/test_cls_2pc_queue.sh... 2023-01-24T10:46:33.551 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_2pc_queue.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_2pc_queue.sh 2023-01-24T10:46:33.621 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:46:33.621 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 22 tests from 1 test suite. 2023-01-24T10:46:33.621 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:46:33.621 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 22 tests from TestCls2PCQueue 2023-01-24T10:46:33.621 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.GetCapacity 2023-01-24T10:46:35.972 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.GetCapacity (2351 ms) 2023-01-24T10:46:35.972 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.AsyncGetCapacity 2023-01-24T10:46:39.000 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.AsyncGetCapacity (3029 ms) 2023-01-24T10:46:39.001 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.Reserve 2023-01-24T10:46:42.006 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.Reserve (3006 ms) 2023-01-24T10:46:42.007 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.AsyncReserve 2023-01-24T10:46:45.047 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.AsyncReserve (3040 ms) 2023-01-24T10:46:45.047 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.Commit 2023-01-24T10:46:48.057 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.Commit (3010 ms) 2023-01-24T10:46:48.057 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.Abort 2023-01-24T10:46:51.082 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.Abort (3025 ms) 2023-01-24T10:46:51.082 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.ReserveError 2023-01-24T10:46:54.086 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.ReserveError (3004 ms) 2023-01-24T10:46:54.086 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.CommitError 2023-01-24T10:46:57.098 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.CommitError (3012 ms) 2023-01-24T10:46:57.098 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.AbortError 2023-01-24T10:47:00.136 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.AbortError (3036 ms) 2023-01-24T10:47:00.137 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.MultiReserve 2023-01-24T10:47:03.157 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.MultiReserve (3023 ms) 2023-01-24T10:47:03.157 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.MultiCommit 2023-01-24T10:47:06.169 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.MultiCommit (3012 ms) 2023-01-24T10:47:06.169 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.MultiAbort 2023-01-24T10:47:09.187 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.MultiAbort (3018 ms) 2023-01-24T10:47:09.187 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.ReserveCommit 2023-01-24T10:47:12.216 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.ReserveCommit (3029 ms) 2023-01-24T10:47:12.216 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.ReserveAbort 2023-01-24T10:47:15.248 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.ReserveAbort (3031 ms) 2023-01-24T10:47:15.248 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.ManualCleanup 2023-01-24T10:47:18.259 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.ManualCleanup (3011 ms) 2023-01-24T10:47:18.259 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.Cleanup 2023-01-24T10:47:21.290 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.Cleanup (3032 ms) 2023-01-24T10:47:21.290 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.MultiProducer 2023-01-24T10:47:42.952 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.MultiProducer (21662 ms) 2023-01-24T10:47:42.952 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.AsyncConsumer 2023-01-24T10:47:46.994 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.AsyncConsumer (4042 ms) 2023-01-24T10:47:46.994 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.MultiProducerConsumer 2023-01-24T10:48:14.963 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.MultiProducerConsumer (27970 ms) 2023-01-24T10:48:14.964 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.ReserveSpillover 2023-01-24T10:48:19.981 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.ReserveSpillover (5018 ms) 2023-01-24T10:48:19.982 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.CommitSpillover 2023-01-24T10:48:26.400 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.CommitSpillover (6419 ms) 2023-01-24T10:48:26.400 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestCls2PCQueue.AbortSpillover 2023-01-24T10:48:33.438 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestCls2PCQueue.AbortSpillover (7037 ms) 2023-01-24T10:48:33.438 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 22 tests from TestCls2PCQueue (119817 ms total) 2023-01-24T10:48:33.438 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:48:33.439 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:48:33.439 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 22 tests from 1 test suite ran. (119817 ms total) 2023-01-24T10:48:33.439 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 22 tests. 2023-01-24T10:48:33.440 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:48:33.441 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:48:33.512 INFO:tasks.workunit:Running workunit cls/test_cls_cas.sh... 2023-01-24T10:48:33.512 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_cas.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cas.sh 2023-01-24T10:48:33.589 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:48:33.590 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 6 tests from 2 test suites. 2023-01-24T10:48:33.590 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:48:33.590 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 5 tests from cls_cas 2023-01-24T10:48:33.590 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_cas.get_put 2023-01-24T10:48:36.447 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_cas.get_put (2857 ms) 2023-01-24T10:48:36.447 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_cas.wrong_put 2023-01-24T10:48:38.970 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_cas.wrong_put (2523 ms) 2023-01-24T10:48:38.970 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_cas.dup_get 2023-01-24T10:48:41.996 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_cas.dup_get (3020 ms) 2023-01-24T10:48:41.997 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_cas.dup_put 2023-01-24T10:48:45.012 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_cas.dup_put (3022 ms) 2023-01-24T10:48:45.012 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_cas.get_wrong_data 2023-01-24T10:48:47.962 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_cas.get_wrong_data (2950 ms) 2023-01-24T10:48:47.963 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 5 tests from cls_cas (14372 ms total) 2023-01-24T10:48:47.963 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:48:47.963 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 1 test from chunk_refs_t 2023-01-24T10:48:47.963 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] chunk_refs_t.size 2023-01-24T10:48:47.963 INFO:tasks.workunit.client.0.smithi078.stdout:1 60 by_object 2023-01-24T10:48:47.964 INFO:tasks.workunit.client.0.smithi078.stdout:2 103 by_object 2023-01-24T10:48:47.964 INFO:tasks.workunit.client.0.smithi078.stdout:4 189 by_object 2023-01-24T10:48:47.964 INFO:tasks.workunit.client.0.smithi078.stdout:8 361 by_object 2023-01-24T10:48:47.964 INFO:tasks.workunit.client.0.smithi078.stdout:16 112 by_hash(32 bits) 2023-01-24T10:48:47.964 INFO:tasks.workunit.client.0.smithi078.stdout:32 208 by_hash(32 bits) 2023-01-24T10:48:47.965 INFO:tasks.workunit.client.0.smithi078.stdout:64 400 by_hash(32 bits) 2023-01-24T10:48:47.965 INFO:tasks.workunit.client.0.smithi078.stdout:128 509 by_hash(10 bits) 2023-01-24T10:48:47.965 INFO:tasks.workunit.client.0.smithi078.stdout:256 499 by_hash(7 bits) 2023-01-24T10:48:47.966 INFO:tasks.workunit.client.0.smithi078.stdout:512 462 by_hash(5 bits) 2023-01-24T10:48:47.966 INFO:tasks.workunit.client.0.smithi078.stdout:1024 466 by_hash(5 bits) 2023-01-24T10:48:47.966 INFO:tasks.workunit.client.0.smithi078.stdout:2048 466 by_hash(5 bits) 2023-01-24T10:48:47.966 INFO:tasks.workunit.client.0.smithi078.stdout:4096 470 by_hash(5 bits) 2023-01-24T10:48:47.967 INFO:tasks.workunit.client.0.smithi078.stdout:8192 498 by_hash(4 bits) 2023-01-24T10:48:47.967 INFO:tasks.workunit.client.0.smithi078.stdout:16384 278 by_hash(3 bits) 2023-01-24T10:48:47.969 INFO:tasks.workunit.client.0.smithi078.stdout:32768 337 by_hash(2 bits) 2023-01-24T10:48:47.974 INFO:tasks.workunit.client.0.smithi078.stdout:65536 367 by_hash(2 bits) 2023-01-24T10:48:47.982 INFO:tasks.workunit.client.0.smithi078.stdout:131072 387 by_hash(1 bits) 2023-01-24T10:48:47.999 INFO:tasks.workunit.client.0.smithi078.stdout:262144 331 by_pool 2023-01-24T10:48:48.032 INFO:tasks.workunit.client.0.smithi078.stdout:524288 21 count 2023-01-24T10:48:48.087 INFO:tasks.workunit.client.0.smithi078.stdout:1048576 21 count 2023-01-24T10:48:48.187 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] chunk_refs_t.size (226 ms) 2023-01-24T10:48:48.188 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 1 test from chunk_refs_t (226 ms total) 2023-01-24T10:48:48.188 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:48:48.188 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:48:48.188 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 6 tests from 2 test suites ran. (14598 ms total) 2023-01-24T10:48:48.188 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 6 tests. 2023-01-24T10:48:48.189 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:48:48.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:48:48.255 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2023-01-24T10:48:48.256 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2023-01-24T10:48:48.327 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:48:48.327 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 31 tests from 1 test suite. 2023-01-24T10:48:48.327 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:48:50.035 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 31 tests from CmpOmap 2023-01-24T10:48:50.036 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2023-01-24T10:48:50.053 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (15 ms) 2023-01-24T10:48:50.053 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2023-01-24T10:48:50.054 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (3 ms) 2023-01-24T10:48:50.054 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2023-01-24T10:48:50.057 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (4 ms) 2023-01-24T10:48:50.057 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2023-01-24T10:48:50.062 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (4 ms) 2023-01-24T10:48:50.062 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_str 2023-01-24T10:48:50.068 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_str (6 ms) 2023-01-24T10:48:50.069 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2023-01-24T10:48:50.076 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_u64 (7 ms) 2023-01-24T10:48:50.076 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2023-01-24T10:48:50.079 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (3 ms) 2023-01-24T10:48:50.079 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2023-01-24T10:48:50.081 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (3 ms) 2023-01-24T10:48:50.082 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2023-01-24T10:48:50.085 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (3 ms) 2023-01-24T10:48:50.086 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2023-01-24T10:48:50.094 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (6 ms) 2023-01-24T10:48:50.094 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2023-01-24T10:48:50.094 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (1 ms) 2023-01-24T10:48:50.094 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2023-01-24T10:48:50.095 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (0 ms) 2023-01-24T10:48:50.095 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2023-01-24T10:48:50.106 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (14 ms) 2023-01-24T10:48:50.106 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2023-01-24T10:48:50.114 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (8 ms) 2023-01-24T10:48:50.114 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2023-01-24T10:48:50.120 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (6 ms) 2023-01-24T10:48:50.120 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2023-01-24T10:48:50.264 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (144 ms) 2023-01-24T10:48:50.264 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2023-01-24T10:48:50.274 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_str (11 ms) 2023-01-24T10:48:50.274 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2023-01-24T10:48:50.294 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (19 ms) 2023-01-24T10:48:50.295 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2023-01-24T10:48:50.297 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (4 ms) 2023-01-24T10:48:50.298 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2023-01-24T10:48:50.300 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (3 ms) 2023-01-24T10:48:50.300 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2023-01-24T10:48:50.301 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (1 ms) 2023-01-24T10:48:50.301 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2023-01-24T10:48:50.301 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (0 ms) 2023-01-24T10:48:50.302 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2023-01-24T10:48:50.313 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (12 ms) 2023-01-24T10:48:50.314 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2023-01-24T10:48:50.327 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (14 ms) 2023-01-24T10:48:50.327 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2023-01-24T10:48:50.345 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (18 ms) 2023-01-24T10:48:50.345 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2023-01-24T10:48:50.352 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (7 ms) 2023-01-24T10:48:50.352 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2023-01-24T10:48:50.355 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (3 ms) 2023-01-24T10:48:50.355 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2023-01-24T10:48:50.357 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (2 ms) 2023-01-24T10:48:50.357 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2023-01-24T10:48:50.358 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (1 ms) 2023-01-24T10:48:50.358 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2023-01-24T10:48:50.359 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (0 ms) 2023-01-24T10:48:50.359 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2023-01-24T10:48:50.379 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (20 ms) 2023-01-24T10:48:50.379 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 31 tests from CmpOmap (343 ms total) 2023-01-24T10:48:50.379 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:48:50.380 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:48:51.049 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 31 tests from 1 test suite ran. (2722 ms total) 2023-01-24T10:48:51.049 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 31 tests. 2023-01-24T10:48:51.051 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:48:51.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:48:51.120 INFO:tasks.workunit:Running workunit cls/test_cls_hello.sh... 2023-01-24T10:48:51.121 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_hello.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_hello.sh 2023-01-24T10:48:51.194 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:48:51.195 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 6 tests from 1 test suite. 2023-01-24T10:48:51.195 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:48:51.195 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 6 tests from ClsHello 2023-01-24T10:48:51.195 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsHello.SayHello 2023-01-24T10:48:54.063 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsHello.SayHello (2869 ms) 2023-01-24T10:48:54.064 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsHello.RecordHello 2023-01-24T10:48:57.096 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsHello.RecordHello (3033 ms) 2023-01-24T10:48:57.097 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsHello.WriteReturnData 2023-01-24T10:48:58.970 INFO:tasks.workunit.client.0.smithi078.stdout:require_osd_release = reef 2023-01-24T10:48:58.989 INFO:tasks.workunit.client.0.smithi078.stdout:0 2023-01-24T10:48:58.990 INFO:tasks.workunit.client.0.smithi078.stdout:1 2023-01-24T10:48:58.991 INFO:tasks.workunit.client.0.smithi078.stdout:2 2023-01-24T10:48:58.993 INFO:tasks.workunit.client.0.smithi078.stdout:3 2023-01-24T10:48:58.994 INFO:tasks.workunit.client.0.smithi078.stdout:4 2023-01-24T10:48:58.995 INFO:tasks.workunit.client.0.smithi078.stdout:5 2023-01-24T10:48:58.996 INFO:tasks.workunit.client.0.smithi078.stdout:6 2023-01-24T10:48:58.999 INFO:tasks.workunit.client.0.smithi078.stdout:7 2023-01-24T10:48:59.002 INFO:tasks.workunit.client.0.smithi078.stdout:8 2023-01-24T10:48:59.003 INFO:tasks.workunit.client.0.smithi078.stdout:9 2023-01-24T10:48:59.005 INFO:tasks.workunit.client.0.smithi078.stdout:10 2023-01-24T10:48:59.006 INFO:tasks.workunit.client.0.smithi078.stdout:11 2023-01-24T10:48:59.009 INFO:tasks.workunit.client.0.smithi078.stdout:12 2023-01-24T10:48:59.011 INFO:tasks.workunit.client.0.smithi078.stdout:13 2023-01-24T10:48:59.014 INFO:tasks.workunit.client.0.smithi078.stdout:14 2023-01-24T10:48:59.016 INFO:tasks.workunit.client.0.smithi078.stdout:15 2023-01-24T10:48:59.018 INFO:tasks.workunit.client.0.smithi078.stdout:16 2023-01-24T10:48:59.019 INFO:tasks.workunit.client.0.smithi078.stdout:17 2023-01-24T10:48:59.022 INFO:tasks.workunit.client.0.smithi078.stdout:18 2023-01-24T10:48:59.025 INFO:tasks.workunit.client.0.smithi078.stdout:19 2023-01-24T10:48:59.028 INFO:tasks.workunit.client.0.smithi078.stdout:20 2023-01-24T10:48:59.030 INFO:tasks.workunit.client.0.smithi078.stdout:21 2023-01-24T10:48:59.032 INFO:tasks.workunit.client.0.smithi078.stdout:22 2023-01-24T10:48:59.035 INFO:tasks.workunit.client.0.smithi078.stdout:23 2023-01-24T10:48:59.038 INFO:tasks.workunit.client.0.smithi078.stdout:24 2023-01-24T10:48:59.042 INFO:tasks.workunit.client.0.smithi078.stdout:25 2023-01-24T10:48:59.043 INFO:tasks.workunit.client.0.smithi078.stdout:26 2023-01-24T10:48:59.043 INFO:tasks.workunit.client.0.smithi078.stdout:27 2023-01-24T10:48:59.045 INFO:tasks.workunit.client.0.smithi078.stdout:28 2023-01-24T10:48:59.048 INFO:tasks.workunit.client.0.smithi078.stdout:29 2023-01-24T10:48:59.051 INFO:tasks.workunit.client.0.smithi078.stdout:30 2023-01-24T10:48:59.053 INFO:tasks.workunit.client.0.smithi078.stdout:31 2023-01-24T10:48:59.056 INFO:tasks.workunit.client.0.smithi078.stdout:32 2023-01-24T10:48:59.058 INFO:tasks.workunit.client.0.smithi078.stdout:33 2023-01-24T10:48:59.059 INFO:tasks.workunit.client.0.smithi078.stdout:34 2023-01-24T10:48:59.060 INFO:tasks.workunit.client.0.smithi078.stdout:35 2023-01-24T10:48:59.061 INFO:tasks.workunit.client.0.smithi078.stdout:36 2023-01-24T10:48:59.062 INFO:tasks.workunit.client.0.smithi078.stdout:37 2023-01-24T10:48:59.063 INFO:tasks.workunit.client.0.smithi078.stdout:38 2023-01-24T10:48:59.064 INFO:tasks.workunit.client.0.smithi078.stdout:39 2023-01-24T10:48:59.064 INFO:tasks.workunit.client.0.smithi078.stdout:40 2023-01-24T10:48:59.065 INFO:tasks.workunit.client.0.smithi078.stdout:41 2023-01-24T10:48:59.069 INFO:tasks.workunit.client.0.smithi078.stdout:42 2023-01-24T10:48:59.071 INFO:tasks.workunit.client.0.smithi078.stdout:43 2023-01-24T10:48:59.074 INFO:tasks.workunit.client.0.smithi078.stdout:44 2023-01-24T10:48:59.075 INFO:tasks.workunit.client.0.smithi078.stdout:45 2023-01-24T10:48:59.078 INFO:tasks.workunit.client.0.smithi078.stdout:46 2023-01-24T10:48:59.081 INFO:tasks.workunit.client.0.smithi078.stdout:47 2023-01-24T10:48:59.084 INFO:tasks.workunit.client.0.smithi078.stdout:48 2023-01-24T10:48:59.087 INFO:tasks.workunit.client.0.smithi078.stdout:49 2023-01-24T10:48:59.087 INFO:tasks.workunit.client.0.smithi078.stdout:50 2023-01-24T10:48:59.089 INFO:tasks.workunit.client.0.smithi078.stdout:51 2023-01-24T10:48:59.090 INFO:tasks.workunit.client.0.smithi078.stdout:52 2023-01-24T10:48:59.093 INFO:tasks.workunit.client.0.smithi078.stdout:53 2023-01-24T10:48:59.094 INFO:tasks.workunit.client.0.smithi078.stdout:54 2023-01-24T10:48:59.095 INFO:tasks.workunit.client.0.smithi078.stdout:55 2023-01-24T10:48:59.096 INFO:tasks.workunit.client.0.smithi078.stdout:56 2023-01-24T10:48:59.097 INFO:tasks.workunit.client.0.smithi078.stdout:57 2023-01-24T10:48:59.098 INFO:tasks.workunit.client.0.smithi078.stdout:58 2023-01-24T10:48:59.099 INFO:tasks.workunit.client.0.smithi078.stdout:59 2023-01-24T10:48:59.100 INFO:tasks.workunit.client.0.smithi078.stdout:60 2023-01-24T10:48:59.101 INFO:tasks.workunit.client.0.smithi078.stdout:61 2023-01-24T10:48:59.102 INFO:tasks.workunit.client.0.smithi078.stdout:62 2023-01-24T10:48:59.103 INFO:tasks.workunit.client.0.smithi078.stdout:63 2023-01-24T10:48:59.104 INFO:tasks.workunit.client.0.smithi078.stdout:64 2023-01-24T10:48:59.105 INFO:tasks.workunit.client.0.smithi078.stdout:65 2023-01-24T10:48:59.106 INFO:tasks.workunit.client.0.smithi078.stdout:66 2023-01-24T10:48:59.107 INFO:tasks.workunit.client.0.smithi078.stdout:67 2023-01-24T10:48:59.108 INFO:tasks.workunit.client.0.smithi078.stdout:68 2023-01-24T10:48:59.109 INFO:tasks.workunit.client.0.smithi078.stdout:69 2023-01-24T10:48:59.110 INFO:tasks.workunit.client.0.smithi078.stdout:70 2023-01-24T10:48:59.111 INFO:tasks.workunit.client.0.smithi078.stdout:71 2023-01-24T10:48:59.112 INFO:tasks.workunit.client.0.smithi078.stdout:72 2023-01-24T10:48:59.113 INFO:tasks.workunit.client.0.smithi078.stdout:73 2023-01-24T10:48:59.114 INFO:tasks.workunit.client.0.smithi078.stdout:74 2023-01-24T10:48:59.115 INFO:tasks.workunit.client.0.smithi078.stdout:75 2023-01-24T10:48:59.116 INFO:tasks.workunit.client.0.smithi078.stdout:76 2023-01-24T10:48:59.117 INFO:tasks.workunit.client.0.smithi078.stdout:77 2023-01-24T10:48:59.118 INFO:tasks.workunit.client.0.smithi078.stdout:78 2023-01-24T10:48:59.119 INFO:tasks.workunit.client.0.smithi078.stdout:79 2023-01-24T10:48:59.120 INFO:tasks.workunit.client.0.smithi078.stdout:80 2023-01-24T10:48:59.121 INFO:tasks.workunit.client.0.smithi078.stdout:81 2023-01-24T10:48:59.122 INFO:tasks.workunit.client.0.smithi078.stdout:82 2023-01-24T10:48:59.123 INFO:tasks.workunit.client.0.smithi078.stdout:83 2023-01-24T10:48:59.124 INFO:tasks.workunit.client.0.smithi078.stdout:84 2023-01-24T10:48:59.125 INFO:tasks.workunit.client.0.smithi078.stdout:85 2023-01-24T10:48:59.126 INFO:tasks.workunit.client.0.smithi078.stdout:86 2023-01-24T10:48:59.127 INFO:tasks.workunit.client.0.smithi078.stdout:87 2023-01-24T10:48:59.128 INFO:tasks.workunit.client.0.smithi078.stdout:88 2023-01-24T10:48:59.129 INFO:tasks.workunit.client.0.smithi078.stdout:89 2023-01-24T10:48:59.130 INFO:tasks.workunit.client.0.smithi078.stdout:90 2023-01-24T10:48:59.131 INFO:tasks.workunit.client.0.smithi078.stdout:91 2023-01-24T10:48:59.132 INFO:tasks.workunit.client.0.smithi078.stdout:92 2023-01-24T10:48:59.133 INFO:tasks.workunit.client.0.smithi078.stdout:93 2023-01-24T10:48:59.134 INFO:tasks.workunit.client.0.smithi078.stdout:94 2023-01-24T10:48:59.136 INFO:tasks.workunit.client.0.smithi078.stdout:95 2023-01-24T10:48:59.139 INFO:tasks.workunit.client.0.smithi078.stdout:96 2023-01-24T10:48:59.142 INFO:tasks.workunit.client.0.smithi078.stdout:97 2023-01-24T10:48:59.145 INFO:tasks.workunit.client.0.smithi078.stdout:98 2023-01-24T10:48:59.147 INFO:tasks.workunit.client.0.smithi078.stdout:99 2023-01-24T10:48:59.148 INFO:tasks.workunit.client.0.smithi078.stdout:100 2023-01-24T10:48:59.151 INFO:tasks.workunit.client.0.smithi078.stdout:101 2023-01-24T10:48:59.153 INFO:tasks.workunit.client.0.smithi078.stdout:102 2023-01-24T10:48:59.156 INFO:tasks.workunit.client.0.smithi078.stdout:103 2023-01-24T10:48:59.159 INFO:tasks.workunit.client.0.smithi078.stdout:104 2023-01-24T10:48:59.161 INFO:tasks.workunit.client.0.smithi078.stdout:105 2023-01-24T10:48:59.164 INFO:tasks.workunit.client.0.smithi078.stdout:106 2023-01-24T10:48:59.165 INFO:tasks.workunit.client.0.smithi078.stdout:107 2023-01-24T10:48:59.167 INFO:tasks.workunit.client.0.smithi078.stdout:108 2023-01-24T10:48:59.170 INFO:tasks.workunit.client.0.smithi078.stdout:109 2023-01-24T10:48:59.172 INFO:tasks.workunit.client.0.smithi078.stdout:110 2023-01-24T10:48:59.174 INFO:tasks.workunit.client.0.smithi078.stdout:111 2023-01-24T10:48:59.177 INFO:tasks.workunit.client.0.smithi078.stdout:112 2023-01-24T10:48:59.178 INFO:tasks.workunit.client.0.smithi078.stdout:113 2023-01-24T10:48:59.179 INFO:tasks.workunit.client.0.smithi078.stdout:114 2023-01-24T10:48:59.180 INFO:tasks.workunit.client.0.smithi078.stdout:115 2023-01-24T10:48:59.182 INFO:tasks.workunit.client.0.smithi078.stdout:116 2023-01-24T10:48:59.182 INFO:tasks.workunit.client.0.smithi078.stdout:117 2023-01-24T10:48:59.183 INFO:tasks.workunit.client.0.smithi078.stdout:118 2023-01-24T10:48:59.184 INFO:tasks.workunit.client.0.smithi078.stdout:119 2023-01-24T10:48:59.185 INFO:tasks.workunit.client.0.smithi078.stdout:120 2023-01-24T10:48:59.186 INFO:tasks.workunit.client.0.smithi078.stdout:121 2023-01-24T10:48:59.187 INFO:tasks.workunit.client.0.smithi078.stdout:122 2023-01-24T10:48:59.188 INFO:tasks.workunit.client.0.smithi078.stdout:123 2023-01-24T10:48:59.189 INFO:tasks.workunit.client.0.smithi078.stdout:124 2023-01-24T10:48:59.192 INFO:tasks.workunit.client.0.smithi078.stdout:125 2023-01-24T10:48:59.195 INFO:tasks.workunit.client.0.smithi078.stdout:126 2023-01-24T10:48:59.198 INFO:tasks.workunit.client.0.smithi078.stdout:127 2023-01-24T10:48:59.199 INFO:tasks.workunit.client.0.smithi078.stdout:128 2023-01-24T10:48:59.201 INFO:tasks.workunit.client.0.smithi078.stdout:129 2023-01-24T10:48:59.204 INFO:tasks.workunit.client.0.smithi078.stdout:130 2023-01-24T10:48:59.207 INFO:tasks.workunit.client.0.smithi078.stdout:131 2023-01-24T10:48:59.208 INFO:tasks.workunit.client.0.smithi078.stdout:132 2023-01-24T10:48:59.209 INFO:tasks.workunit.client.0.smithi078.stdout:133 2023-01-24T10:48:59.210 INFO:tasks.workunit.client.0.smithi078.stdout:134 2023-01-24T10:48:59.211 INFO:tasks.workunit.client.0.smithi078.stdout:135 2023-01-24T10:48:59.212 INFO:tasks.workunit.client.0.smithi078.stdout:136 2023-01-24T10:48:59.213 INFO:tasks.workunit.client.0.smithi078.stdout:137 2023-01-24T10:48:59.214 INFO:tasks.workunit.client.0.smithi078.stdout:138 2023-01-24T10:48:59.214 INFO:tasks.workunit.client.0.smithi078.stdout:139 2023-01-24T10:48:59.215 INFO:tasks.workunit.client.0.smithi078.stdout:140 2023-01-24T10:48:59.216 INFO:tasks.workunit.client.0.smithi078.stdout:141 2023-01-24T10:48:59.217 INFO:tasks.workunit.client.0.smithi078.stdout:142 2023-01-24T10:48:59.218 INFO:tasks.workunit.client.0.smithi078.stdout:143 2023-01-24T10:48:59.219 INFO:tasks.workunit.client.0.smithi078.stdout:144 2023-01-24T10:48:59.219 INFO:tasks.workunit.client.0.smithi078.stdout:145 2023-01-24T10:48:59.220 INFO:tasks.workunit.client.0.smithi078.stdout:146 2023-01-24T10:48:59.223 INFO:tasks.workunit.client.0.smithi078.stdout:147 2023-01-24T10:48:59.226 INFO:tasks.workunit.client.0.smithi078.stdout:148 2023-01-24T10:48:59.227 INFO:tasks.workunit.client.0.smithi078.stdout:149 2023-01-24T10:48:59.230 INFO:tasks.workunit.client.0.smithi078.stdout:150 2023-01-24T10:48:59.232 INFO:tasks.workunit.client.0.smithi078.stdout:151 2023-01-24T10:48:59.235 INFO:tasks.workunit.client.0.smithi078.stdout:152 2023-01-24T10:48:59.238 INFO:tasks.workunit.client.0.smithi078.stdout:153 2023-01-24T10:48:59.241 INFO:tasks.workunit.client.0.smithi078.stdout:154 2023-01-24T10:48:59.242 INFO:tasks.workunit.client.0.smithi078.stdout:155 2023-01-24T10:48:59.243 INFO:tasks.workunit.client.0.smithi078.stdout:156 2023-01-24T10:48:59.244 INFO:tasks.workunit.client.0.smithi078.stdout:157 2023-01-24T10:48:59.245 INFO:tasks.workunit.client.0.smithi078.stdout:158 2023-01-24T10:48:59.246 INFO:tasks.workunit.client.0.smithi078.stdout:159 2023-01-24T10:48:59.247 INFO:tasks.workunit.client.0.smithi078.stdout:160 2023-01-24T10:48:59.248 INFO:tasks.workunit.client.0.smithi078.stdout:161 2023-01-24T10:48:59.249 INFO:tasks.workunit.client.0.smithi078.stdout:162 2023-01-24T10:48:59.250 INFO:tasks.workunit.client.0.smithi078.stdout:163 2023-01-24T10:48:59.251 INFO:tasks.workunit.client.0.smithi078.stdout:164 2023-01-24T10:48:59.254 INFO:tasks.workunit.client.0.smithi078.stdout:165 2023-01-24T10:48:59.256 INFO:tasks.workunit.client.0.smithi078.stdout:166 2023-01-24T10:48:59.259 INFO:tasks.workunit.client.0.smithi078.stdout:167 2023-01-24T10:48:59.262 INFO:tasks.workunit.client.0.smithi078.stdout:168 2023-01-24T10:48:59.264 INFO:tasks.workunit.client.0.smithi078.stdout:169 2023-01-24T10:48:59.267 INFO:tasks.workunit.client.0.smithi078.stdout:170 2023-01-24T10:48:59.268 INFO:tasks.workunit.client.0.smithi078.stdout:171 2023-01-24T10:48:59.271 INFO:tasks.workunit.client.0.smithi078.stdout:172 2023-01-24T10:48:59.274 INFO:tasks.workunit.client.0.smithi078.stdout:173 2023-01-24T10:48:59.277 INFO:tasks.workunit.client.0.smithi078.stdout:174 2023-01-24T10:48:59.279 INFO:tasks.workunit.client.0.smithi078.stdout:175 2023-01-24T10:48:59.282 INFO:tasks.workunit.client.0.smithi078.stdout:176 2023-01-24T10:48:59.285 INFO:tasks.workunit.client.0.smithi078.stdout:177 2023-01-24T10:48:59.286 INFO:tasks.workunit.client.0.smithi078.stdout:178 2023-01-24T10:48:59.291 INFO:tasks.workunit.client.0.smithi078.stdout:179 2023-01-24T10:48:59.293 INFO:tasks.workunit.client.0.smithi078.stdout:180 2023-01-24T10:48:59.294 INFO:tasks.workunit.client.0.smithi078.stdout:181 2023-01-24T10:48:59.295 INFO:tasks.workunit.client.0.smithi078.stdout:182 2023-01-24T10:48:59.297 INFO:tasks.workunit.client.0.smithi078.stdout:183 2023-01-24T10:48:59.300 INFO:tasks.workunit.client.0.smithi078.stdout:184 2023-01-24T10:48:59.302 INFO:tasks.workunit.client.0.smithi078.stdout:185 2023-01-24T10:48:59.307 INFO:tasks.workunit.client.0.smithi078.stdout:186 2023-01-24T10:48:59.310 INFO:tasks.workunit.client.0.smithi078.stdout:187 2023-01-24T10:48:59.311 INFO:tasks.workunit.client.0.smithi078.stdout:188 2023-01-24T10:48:59.314 INFO:tasks.workunit.client.0.smithi078.stdout:189 2023-01-24T10:48:59.317 INFO:tasks.workunit.client.0.smithi078.stdout:190 2023-01-24T10:48:59.319 INFO:tasks.workunit.client.0.smithi078.stdout:191 2023-01-24T10:48:59.323 INFO:tasks.workunit.client.0.smithi078.stdout:192 2023-01-24T10:48:59.324 INFO:tasks.workunit.client.0.smithi078.stdout:193 2023-01-24T10:48:59.326 INFO:tasks.workunit.client.0.smithi078.stdout:194 2023-01-24T10:48:59.329 INFO:tasks.workunit.client.0.smithi078.stdout:195 2023-01-24T10:48:59.331 INFO:tasks.workunit.client.0.smithi078.stdout:196 2023-01-24T10:48:59.334 INFO:tasks.workunit.client.0.smithi078.stdout:197 2023-01-24T10:48:59.337 INFO:tasks.workunit.client.0.smithi078.stdout:198 2023-01-24T10:48:59.339 INFO:tasks.workunit.client.0.smithi078.stdout:199 2023-01-24T10:48:59.340 INFO:tasks.workunit.client.0.smithi078.stdout:200 2023-01-24T10:48:59.341 INFO:tasks.workunit.client.0.smithi078.stdout:201 2023-01-24T10:48:59.342 INFO:tasks.workunit.client.0.smithi078.stdout:202 2023-01-24T10:48:59.343 INFO:tasks.workunit.client.0.smithi078.stdout:203 2023-01-24T10:48:59.344 INFO:tasks.workunit.client.0.smithi078.stdout:204 2023-01-24T10:48:59.345 INFO:tasks.workunit.client.0.smithi078.stdout:205 2023-01-24T10:48:59.345 INFO:tasks.workunit.client.0.smithi078.stdout:206 2023-01-24T10:48:59.346 INFO:tasks.workunit.client.0.smithi078.stdout:207 2023-01-24T10:48:59.347 INFO:tasks.workunit.client.0.smithi078.stdout:208 2023-01-24T10:48:59.348 INFO:tasks.workunit.client.0.smithi078.stdout:209 2023-01-24T10:48:59.349 INFO:tasks.workunit.client.0.smithi078.stdout:210 2023-01-24T10:48:59.350 INFO:tasks.workunit.client.0.smithi078.stdout:211 2023-01-24T10:48:59.351 INFO:tasks.workunit.client.0.smithi078.stdout:212 2023-01-24T10:48:59.352 INFO:tasks.workunit.client.0.smithi078.stdout:213 2023-01-24T10:48:59.353 INFO:tasks.workunit.client.0.smithi078.stdout:214 2023-01-24T10:48:59.354 INFO:tasks.workunit.client.0.smithi078.stdout:215 2023-01-24T10:48:59.355 INFO:tasks.workunit.client.0.smithi078.stdout:216 2023-01-24T10:48:59.355 INFO:tasks.workunit.client.0.smithi078.stdout:217 2023-01-24T10:48:59.356 INFO:tasks.workunit.client.0.smithi078.stdout:218 2023-01-24T10:48:59.357 INFO:tasks.workunit.client.0.smithi078.stdout:219 2023-01-24T10:48:59.358 INFO:tasks.workunit.client.0.smithi078.stdout:220 2023-01-24T10:48:59.359 INFO:tasks.workunit.client.0.smithi078.stdout:221 2023-01-24T10:48:59.360 INFO:tasks.workunit.client.0.smithi078.stdout:222 2023-01-24T10:48:59.361 INFO:tasks.workunit.client.0.smithi078.stdout:223 2023-01-24T10:48:59.362 INFO:tasks.workunit.client.0.smithi078.stdout:224 2023-01-24T10:48:59.363 INFO:tasks.workunit.client.0.smithi078.stdout:225 2023-01-24T10:48:59.364 INFO:tasks.workunit.client.0.smithi078.stdout:226 2023-01-24T10:48:59.365 INFO:tasks.workunit.client.0.smithi078.stdout:227 2023-01-24T10:48:59.366 INFO:tasks.workunit.client.0.smithi078.stdout:228 2023-01-24T10:48:59.367 INFO:tasks.workunit.client.0.smithi078.stdout:229 2023-01-24T10:48:59.368 INFO:tasks.workunit.client.0.smithi078.stdout:230 2023-01-24T10:48:59.369 INFO:tasks.workunit.client.0.smithi078.stdout:231 2023-01-24T10:48:59.370 INFO:tasks.workunit.client.0.smithi078.stdout:232 2023-01-24T10:48:59.371 INFO:tasks.workunit.client.0.smithi078.stdout:233 2023-01-24T10:48:59.372 INFO:tasks.workunit.client.0.smithi078.stdout:234 2023-01-24T10:48:59.373 INFO:tasks.workunit.client.0.smithi078.stdout:235 2023-01-24T10:48:59.374 INFO:tasks.workunit.client.0.smithi078.stdout:236 2023-01-24T10:48:59.375 INFO:tasks.workunit.client.0.smithi078.stdout:237 2023-01-24T10:48:59.376 INFO:tasks.workunit.client.0.smithi078.stdout:238 2023-01-24T10:48:59.378 INFO:tasks.workunit.client.0.smithi078.stdout:239 2023-01-24T10:48:59.379 INFO:tasks.workunit.client.0.smithi078.stdout:240 2023-01-24T10:48:59.379 INFO:tasks.workunit.client.0.smithi078.stdout:241 2023-01-24T10:48:59.380 INFO:tasks.workunit.client.0.smithi078.stdout:242 2023-01-24T10:48:59.381 INFO:tasks.workunit.client.0.smithi078.stdout:243 2023-01-24T10:48:59.382 INFO:tasks.workunit.client.0.smithi078.stdout:244 2023-01-24T10:48:59.383 INFO:tasks.workunit.client.0.smithi078.stdout:245 2023-01-24T10:48:59.384 INFO:tasks.workunit.client.0.smithi078.stdout:246 2023-01-24T10:48:59.385 INFO:tasks.workunit.client.0.smithi078.stdout:247 2023-01-24T10:48:59.386 INFO:tasks.workunit.client.0.smithi078.stdout:248 2023-01-24T10:48:59.387 INFO:tasks.workunit.client.0.smithi078.stdout:249 2023-01-24T10:48:59.388 INFO:tasks.workunit.client.0.smithi078.stdout:250 2023-01-24T10:48:59.388 INFO:tasks.workunit.client.0.smithi078.stdout:251 2023-01-24T10:48:59.390 INFO:tasks.workunit.client.0.smithi078.stdout:252 2023-01-24T10:48:59.392 INFO:tasks.workunit.client.0.smithi078.stdout:253 2023-01-24T10:48:59.395 INFO:tasks.workunit.client.0.smithi078.stdout:254 2023-01-24T10:48:59.398 INFO:tasks.workunit.client.0.smithi078.stdout:255 2023-01-24T10:48:59.401 INFO:tasks.workunit.client.0.smithi078.stdout:256 2023-01-24T10:48:59.402 INFO:tasks.workunit.client.0.smithi078.stdout:257 2023-01-24T10:48:59.405 INFO:tasks.workunit.client.0.smithi078.stdout:258 2023-01-24T10:48:59.409 INFO:tasks.workunit.client.0.smithi078.stdout:259 2023-01-24T10:48:59.410 INFO:tasks.workunit.client.0.smithi078.stdout:260 2023-01-24T10:48:59.413 INFO:tasks.workunit.client.0.smithi078.stdout:261 2023-01-24T10:48:59.416 INFO:tasks.workunit.client.0.smithi078.stdout:262 2023-01-24T10:48:59.419 INFO:tasks.workunit.client.0.smithi078.stdout:263 2023-01-24T10:48:59.423 INFO:tasks.workunit.client.0.smithi078.stdout:264 2023-01-24T10:48:59.423 INFO:tasks.workunit.client.0.smithi078.stdout:265 2023-01-24T10:48:59.425 INFO:tasks.workunit.client.0.smithi078.stdout:266 2023-01-24T10:48:59.430 INFO:tasks.workunit.client.0.smithi078.stdout:267 2023-01-24T10:48:59.431 INFO:tasks.workunit.client.0.smithi078.stdout:268 2023-01-24T10:48:59.433 INFO:tasks.workunit.client.0.smithi078.stdout:269 2023-01-24T10:48:59.434 INFO:tasks.workunit.client.0.smithi078.stdout:270 2023-01-24T10:48:59.435 INFO:tasks.workunit.client.0.smithi078.stdout:271 2023-01-24T10:48:59.436 INFO:tasks.workunit.client.0.smithi078.stdout:272 2023-01-24T10:48:59.437 INFO:tasks.workunit.client.0.smithi078.stdout:273 2023-01-24T10:48:59.438 INFO:tasks.workunit.client.0.smithi078.stdout:274 2023-01-24T10:48:59.439 INFO:tasks.workunit.client.0.smithi078.stdout:275 2023-01-24T10:48:59.440 INFO:tasks.workunit.client.0.smithi078.stdout:276 2023-01-24T10:48:59.440 INFO:tasks.workunit.client.0.smithi078.stdout:277 2023-01-24T10:48:59.441 INFO:tasks.workunit.client.0.smithi078.stdout:278 2023-01-24T10:48:59.442 INFO:tasks.workunit.client.0.smithi078.stdout:279 2023-01-24T10:48:59.443 INFO:tasks.workunit.client.0.smithi078.stdout:280 2023-01-24T10:48:59.444 INFO:tasks.workunit.client.0.smithi078.stdout:281 2023-01-24T10:48:59.445 INFO:tasks.workunit.client.0.smithi078.stdout:282 2023-01-24T10:48:59.446 INFO:tasks.workunit.client.0.smithi078.stdout:283 2023-01-24T10:48:59.447 INFO:tasks.workunit.client.0.smithi078.stdout:284 2023-01-24T10:48:59.448 INFO:tasks.workunit.client.0.smithi078.stdout:285 2023-01-24T10:48:59.449 INFO:tasks.workunit.client.0.smithi078.stdout:286 2023-01-24T10:48:59.450 INFO:tasks.workunit.client.0.smithi078.stdout:287 2023-01-24T10:48:59.450 INFO:tasks.workunit.client.0.smithi078.stdout:288 2023-01-24T10:48:59.451 INFO:tasks.workunit.client.0.smithi078.stdout:289 2023-01-24T10:48:59.453 INFO:tasks.workunit.client.0.smithi078.stdout:290 2023-01-24T10:48:59.453 INFO:tasks.workunit.client.0.smithi078.stdout:291 2023-01-24T10:48:59.454 INFO:tasks.workunit.client.0.smithi078.stdout:292 2023-01-24T10:48:59.455 INFO:tasks.workunit.client.0.smithi078.stdout:293 2023-01-24T10:48:59.456 INFO:tasks.workunit.client.0.smithi078.stdout:294 2023-01-24T10:48:59.457 INFO:tasks.workunit.client.0.smithi078.stdout:295 2023-01-24T10:48:59.458 INFO:tasks.workunit.client.0.smithi078.stdout:296 2023-01-24T10:48:59.459 INFO:tasks.workunit.client.0.smithi078.stdout:297 2023-01-24T10:48:59.460 INFO:tasks.workunit.client.0.smithi078.stdout:298 2023-01-24T10:48:59.460 INFO:tasks.workunit.client.0.smithi078.stdout:299 2023-01-24T10:48:59.461 INFO:tasks.workunit.client.0.smithi078.stdout:300 2023-01-24T10:48:59.462 INFO:tasks.workunit.client.0.smithi078.stdout:301 2023-01-24T10:48:59.463 INFO:tasks.workunit.client.0.smithi078.stdout:302 2023-01-24T10:48:59.464 INFO:tasks.workunit.client.0.smithi078.stdout:303 2023-01-24T10:48:59.465 INFO:tasks.workunit.client.0.smithi078.stdout:304 2023-01-24T10:48:59.466 INFO:tasks.workunit.client.0.smithi078.stdout:305 2023-01-24T10:48:59.467 INFO:tasks.workunit.client.0.smithi078.stdout:306 2023-01-24T10:48:59.468 INFO:tasks.workunit.client.0.smithi078.stdout:307 2023-01-24T10:48:59.469 INFO:tasks.workunit.client.0.smithi078.stdout:308 2023-01-24T10:48:59.469 INFO:tasks.workunit.client.0.smithi078.stdout:309 2023-01-24T10:48:59.470 INFO:tasks.workunit.client.0.smithi078.stdout:310 2023-01-24T10:48:59.471 INFO:tasks.workunit.client.0.smithi078.stdout:311 2023-01-24T10:48:59.472 INFO:tasks.workunit.client.0.smithi078.stdout:312 2023-01-24T10:48:59.473 INFO:tasks.workunit.client.0.smithi078.stdout:313 2023-01-24T10:48:59.474 INFO:tasks.workunit.client.0.smithi078.stdout:314 2023-01-24T10:48:59.475 INFO:tasks.workunit.client.0.smithi078.stdout:315 2023-01-24T10:48:59.476 INFO:tasks.workunit.client.0.smithi078.stdout:316 2023-01-24T10:48:59.477 INFO:tasks.workunit.client.0.smithi078.stdout:317 2023-01-24T10:48:59.478 INFO:tasks.workunit.client.0.smithi078.stdout:318 2023-01-24T10:48:59.479 INFO:tasks.workunit.client.0.smithi078.stdout:319 2023-01-24T10:48:59.480 INFO:tasks.workunit.client.0.smithi078.stdout:320 2023-01-24T10:48:59.480 INFO:tasks.workunit.client.0.smithi078.stdout:321 2023-01-24T10:48:59.481 INFO:tasks.workunit.client.0.smithi078.stdout:322 2023-01-24T10:48:59.484 INFO:tasks.workunit.client.0.smithi078.stdout:323 2023-01-24T10:48:59.487 INFO:tasks.workunit.client.0.smithi078.stdout:324 2023-01-24T10:48:59.489 INFO:tasks.workunit.client.0.smithi078.stdout:325 2023-01-24T10:48:59.492 INFO:tasks.workunit.client.0.smithi078.stdout:326 2023-01-24T10:48:59.495 INFO:tasks.workunit.client.0.smithi078.stdout:327 2023-01-24T10:48:59.496 INFO:tasks.workunit.client.0.smithi078.stdout:328 2023-01-24T10:48:59.498 INFO:tasks.workunit.client.0.smithi078.stdout:329 2023-01-24T10:48:59.501 INFO:tasks.workunit.client.0.smithi078.stdout:330 2023-01-24T10:48:59.504 INFO:tasks.workunit.client.0.smithi078.stdout:331 2023-01-24T10:48:59.506 INFO:tasks.workunit.client.0.smithi078.stdout:332 2023-01-24T10:48:59.509 INFO:tasks.workunit.client.0.smithi078.stdout:333 2023-01-24T10:48:59.512 INFO:tasks.workunit.client.0.smithi078.stdout:334 2023-01-24T10:48:59.513 INFO:tasks.workunit.client.0.smithi078.stdout:335 2023-01-24T10:48:59.516 INFO:tasks.workunit.client.0.smithi078.stdout:336 2023-01-24T10:48:59.520 INFO:tasks.workunit.client.0.smithi078.stdout:337 2023-01-24T10:48:59.521 INFO:tasks.workunit.client.0.smithi078.stdout:338 2023-01-24T10:48:59.523 INFO:tasks.workunit.client.0.smithi078.stdout:339 2023-01-24T10:48:59.526 INFO:tasks.workunit.client.0.smithi078.stdout:340 2023-01-24T10:48:59.529 INFO:tasks.workunit.client.0.smithi078.stdout:341 2023-01-24T10:48:59.532 INFO:tasks.workunit.client.0.smithi078.stdout:342 2023-01-24T10:48:59.535 INFO:tasks.workunit.client.0.smithi078.stdout:343 2023-01-24T10:48:59.537 INFO:tasks.workunit.client.0.smithi078.stdout:344 2023-01-24T10:48:59.538 INFO:tasks.workunit.client.0.smithi078.stdout:345 2023-01-24T10:48:59.541 INFO:tasks.workunit.client.0.smithi078.stdout:346 2023-01-24T10:48:59.544 INFO:tasks.workunit.client.0.smithi078.stdout:347 2023-01-24T10:48:59.545 INFO:tasks.workunit.client.0.smithi078.stdout:348 2023-01-24T10:48:59.546 INFO:tasks.workunit.client.0.smithi078.stdout:349 2023-01-24T10:48:59.547 INFO:tasks.workunit.client.0.smithi078.stdout:350 2023-01-24T10:48:59.548 INFO:tasks.workunit.client.0.smithi078.stdout:351 2023-01-24T10:48:59.549 INFO:tasks.workunit.client.0.smithi078.stdout:352 2023-01-24T10:48:59.550 INFO:tasks.workunit.client.0.smithi078.stdout:353 2023-01-24T10:48:59.551 INFO:tasks.workunit.client.0.smithi078.stdout:354 2023-01-24T10:48:59.555 INFO:tasks.workunit.client.0.smithi078.stdout:355 2023-01-24T10:48:59.557 INFO:tasks.workunit.client.0.smithi078.stdout:356 2023-01-24T10:48:59.560 INFO:tasks.workunit.client.0.smithi078.stdout:357 2023-01-24T10:48:59.563 INFO:tasks.workunit.client.0.smithi078.stdout:358 2023-01-24T10:48:59.566 INFO:tasks.workunit.client.0.smithi078.stdout:359 2023-01-24T10:48:59.568 INFO:tasks.workunit.client.0.smithi078.stdout:360 2023-01-24T10:48:59.571 INFO:tasks.workunit.client.0.smithi078.stdout:361 2023-01-24T10:48:59.574 INFO:tasks.workunit.client.0.smithi078.stdout:362 2023-01-24T10:48:59.577 INFO:tasks.workunit.client.0.smithi078.stdout:363 2023-01-24T10:48:59.580 INFO:tasks.workunit.client.0.smithi078.stdout:364 2023-01-24T10:48:59.582 INFO:tasks.workunit.client.0.smithi078.stdout:365 2023-01-24T10:48:59.585 INFO:tasks.workunit.client.0.smithi078.stdout:366 2023-01-24T10:48:59.586 INFO:tasks.workunit.client.0.smithi078.stdout:367 2023-01-24T10:48:59.587 INFO:tasks.workunit.client.0.smithi078.stdout:368 2023-01-24T10:48:59.588 INFO:tasks.workunit.client.0.smithi078.stdout:369 2023-01-24T10:48:59.589 INFO:tasks.workunit.client.0.smithi078.stdout:370 2023-01-24T10:48:59.590 INFO:tasks.workunit.client.0.smithi078.stdout:371 2023-01-24T10:48:59.591 INFO:tasks.workunit.client.0.smithi078.stdout:372 2023-01-24T10:48:59.594 INFO:tasks.workunit.client.0.smithi078.stdout:373 2023-01-24T10:48:59.594 INFO:tasks.workunit.client.0.smithi078.stdout:374 2023-01-24T10:48:59.594 INFO:tasks.workunit.client.0.smithi078.stdout:375 2023-01-24T10:48:59.595 INFO:tasks.workunit.client.0.smithi078.stdout:376 2023-01-24T10:48:59.596 INFO:tasks.workunit.client.0.smithi078.stdout:377 2023-01-24T10:48:59.597 INFO:tasks.workunit.client.0.smithi078.stdout:378 2023-01-24T10:48:59.598 INFO:tasks.workunit.client.0.smithi078.stdout:379 2023-01-24T10:48:59.599 INFO:tasks.workunit.client.0.smithi078.stdout:380 2023-01-24T10:48:59.600 INFO:tasks.workunit.client.0.smithi078.stdout:381 2023-01-24T10:48:59.601 INFO:tasks.workunit.client.0.smithi078.stdout:382 2023-01-24T10:48:59.603 INFO:tasks.workunit.client.0.smithi078.stdout:383 2023-01-24T10:48:59.606 INFO:tasks.workunit.client.0.smithi078.stdout:384 2023-01-24T10:48:59.611 INFO:tasks.workunit.client.0.smithi078.stdout:385 2023-01-24T10:48:59.611 INFO:tasks.workunit.client.0.smithi078.stdout:386 2023-01-24T10:48:59.614 INFO:tasks.workunit.client.0.smithi078.stdout:387 2023-01-24T10:48:59.615 INFO:tasks.workunit.client.0.smithi078.stdout:388 2023-01-24T10:48:59.616 INFO:tasks.workunit.client.0.smithi078.stdout:389 2023-01-24T10:48:59.617 INFO:tasks.workunit.client.0.smithi078.stdout:390 2023-01-24T10:48:59.618 INFO:tasks.workunit.client.0.smithi078.stdout:391 2023-01-24T10:48:59.619 INFO:tasks.workunit.client.0.smithi078.stdout:392 2023-01-24T10:48:59.622 INFO:tasks.workunit.client.0.smithi078.stdout:393 2023-01-24T10:48:59.625 INFO:tasks.workunit.client.0.smithi078.stdout:394 2023-01-24T10:48:59.627 INFO:tasks.workunit.client.0.smithi078.stdout:395 2023-01-24T10:48:59.630 INFO:tasks.workunit.client.0.smithi078.stdout:396 2023-01-24T10:48:59.633 INFO:tasks.workunit.client.0.smithi078.stdout:397 2023-01-24T10:48:59.636 INFO:tasks.workunit.client.0.smithi078.stdout:398 2023-01-24T10:48:59.638 INFO:tasks.workunit.client.0.smithi078.stdout:399 2023-01-24T10:48:59.641 INFO:tasks.workunit.client.0.smithi078.stdout:400 2023-01-24T10:48:59.644 INFO:tasks.workunit.client.0.smithi078.stdout:401 2023-01-24T10:48:59.647 INFO:tasks.workunit.client.0.smithi078.stdout:402 2023-01-24T10:48:59.650 INFO:tasks.workunit.client.0.smithi078.stdout:403 2023-01-24T10:48:59.652 INFO:tasks.workunit.client.0.smithi078.stdout:404 2023-01-24T10:48:59.655 INFO:tasks.workunit.client.0.smithi078.stdout:405 2023-01-24T10:48:59.657 INFO:tasks.workunit.client.0.smithi078.stdout:406 2023-01-24T10:48:59.658 INFO:tasks.workunit.client.0.smithi078.stdout:407 2023-01-24T10:48:59.661 INFO:tasks.workunit.client.0.smithi078.stdout:408 2023-01-24T10:48:59.663 INFO:tasks.workunit.client.0.smithi078.stdout:409 2023-01-24T10:48:59.666 INFO:tasks.workunit.client.0.smithi078.stdout:410 2023-01-24T10:48:59.667 INFO:tasks.workunit.client.0.smithi078.stdout:411 2023-01-24T10:48:59.668 INFO:tasks.workunit.client.0.smithi078.stdout:412 2023-01-24T10:48:59.671 INFO:tasks.workunit.client.0.smithi078.stdout:413 2023-01-24T10:48:59.671 INFO:tasks.workunit.client.0.smithi078.stdout:414 2023-01-24T10:48:59.672 INFO:tasks.workunit.client.0.smithi078.stdout:415 2023-01-24T10:48:59.672 INFO:tasks.workunit.client.0.smithi078.stdout:416 2023-01-24T10:48:59.674 INFO:tasks.workunit.client.0.smithi078.stdout:417 2023-01-24T10:48:59.675 INFO:tasks.workunit.client.0.smithi078.stdout:418 2023-01-24T10:48:59.675 INFO:tasks.workunit.client.0.smithi078.stdout:419 2023-01-24T10:48:59.678 INFO:tasks.workunit.client.0.smithi078.stdout:420 2023-01-24T10:48:59.679 INFO:tasks.workunit.client.0.smithi078.stdout:421 2023-01-24T10:48:59.679 INFO:tasks.workunit.client.0.smithi078.stdout:422 2023-01-24T10:48:59.680 INFO:tasks.workunit.client.0.smithi078.stdout:423 2023-01-24T10:48:59.685 INFO:tasks.workunit.client.0.smithi078.stdout:424 2023-01-24T10:48:59.685 INFO:tasks.workunit.client.0.smithi078.stdout:425 2023-01-24T10:48:59.685 INFO:tasks.workunit.client.0.smithi078.stdout:426 2023-01-24T10:48:59.686 INFO:tasks.workunit.client.0.smithi078.stdout:427 2023-01-24T10:48:59.686 INFO:tasks.workunit.client.0.smithi078.stdout:428 2023-01-24T10:48:59.686 INFO:tasks.workunit.client.0.smithi078.stdout:429 2023-01-24T10:48:59.687 INFO:tasks.workunit.client.0.smithi078.stdout:430 2023-01-24T10:48:59.687 INFO:tasks.workunit.client.0.smithi078.stdout:431 2023-01-24T10:48:59.689 INFO:tasks.workunit.client.0.smithi078.stdout:432 2023-01-24T10:48:59.689 INFO:tasks.workunit.client.0.smithi078.stdout:433 2023-01-24T10:48:59.690 INFO:tasks.workunit.client.0.smithi078.stdout:434 2023-01-24T10:48:59.691 INFO:tasks.workunit.client.0.smithi078.stdout:435 2023-01-24T10:48:59.693 INFO:tasks.workunit.client.0.smithi078.stdout:436 2023-01-24T10:48:59.693 INFO:tasks.workunit.client.0.smithi078.stdout:437 2023-01-24T10:48:59.695 INFO:tasks.workunit.client.0.smithi078.stdout:438 2023-01-24T10:48:59.696 INFO:tasks.workunit.client.0.smithi078.stdout:439 2023-01-24T10:48:59.697 INFO:tasks.workunit.client.0.smithi078.stdout:440 2023-01-24T10:48:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:441 2023-01-24T10:48:59.698 INFO:tasks.workunit.client.0.smithi078.stdout:442 2023-01-24T10:48:59.699 INFO:tasks.workunit.client.0.smithi078.stdout:443 2023-01-24T10:48:59.700 INFO:tasks.workunit.client.0.smithi078.stdout:444 2023-01-24T10:48:59.701 INFO:tasks.workunit.client.0.smithi078.stdout:445 2023-01-24T10:48:59.702 INFO:tasks.workunit.client.0.smithi078.stdout:446 2023-01-24T10:48:59.703 INFO:tasks.workunit.client.0.smithi078.stdout:447 2023-01-24T10:48:59.704 INFO:tasks.workunit.client.0.smithi078.stdout:448 2023-01-24T10:48:59.705 INFO:tasks.workunit.client.0.smithi078.stdout:449 2023-01-24T10:48:59.706 INFO:tasks.workunit.client.0.smithi078.stdout:450 2023-01-24T10:48:59.707 INFO:tasks.workunit.client.0.smithi078.stdout:451 2023-01-24T10:48:59.707 INFO:tasks.workunit.client.0.smithi078.stdout:452 2023-01-24T10:48:59.708 INFO:tasks.workunit.client.0.smithi078.stdout:453 2023-01-24T10:48:59.709 INFO:tasks.workunit.client.0.smithi078.stdout:454 2023-01-24T10:48:59.710 INFO:tasks.workunit.client.0.smithi078.stdout:455 2023-01-24T10:48:59.711 INFO:tasks.workunit.client.0.smithi078.stdout:456 2023-01-24T10:48:59.712 INFO:tasks.workunit.client.0.smithi078.stdout:457 2023-01-24T10:48:59.713 INFO:tasks.workunit.client.0.smithi078.stdout:458 2023-01-24T10:48:59.716 INFO:tasks.workunit.client.0.smithi078.stdout:459 2023-01-24T10:48:59.719 INFO:tasks.workunit.client.0.smithi078.stdout:460 2023-01-24T10:48:59.721 INFO:tasks.workunit.client.0.smithi078.stdout:461 2023-01-24T10:48:59.724 INFO:tasks.workunit.client.0.smithi078.stdout:462 2023-01-24T10:48:59.727 INFO:tasks.workunit.client.0.smithi078.stdout:463 2023-01-24T10:48:59.729 INFO:tasks.workunit.client.0.smithi078.stdout:464 2023-01-24T10:48:59.730 INFO:tasks.workunit.client.0.smithi078.stdout:465 2023-01-24T10:48:59.731 INFO:tasks.workunit.client.0.smithi078.stdout:466 2023-01-24T10:48:59.732 INFO:tasks.workunit.client.0.smithi078.stdout:467 2023-01-24T10:48:59.733 INFO:tasks.workunit.client.0.smithi078.stdout:468 2023-01-24T10:48:59.734 INFO:tasks.workunit.client.0.smithi078.stdout:469 2023-01-24T10:48:59.734 INFO:tasks.workunit.client.0.smithi078.stdout:470 2023-01-24T10:48:59.736 INFO:tasks.workunit.client.0.smithi078.stdout:471 2023-01-24T10:48:59.739 INFO:tasks.workunit.client.0.smithi078.stdout:472 2023-01-24T10:48:59.743 INFO:tasks.workunit.client.0.smithi078.stdout:473 2023-01-24T10:48:59.745 INFO:tasks.workunit.client.0.smithi078.stdout:474 2023-01-24T10:48:59.747 INFO:tasks.workunit.client.0.smithi078.stdout:475 2023-01-24T10:48:59.750 INFO:tasks.workunit.client.0.smithi078.stdout:476 2023-01-24T10:48:59.754 INFO:tasks.workunit.client.0.smithi078.stdout:477 2023-01-24T10:48:59.755 INFO:tasks.workunit.client.0.smithi078.stdout:478 2023-01-24T10:48:59.759 INFO:tasks.workunit.client.0.smithi078.stdout:479 2023-01-24T10:48:59.761 INFO:tasks.workunit.client.0.smithi078.stdout:480 2023-01-24T10:48:59.763 INFO:tasks.workunit.client.0.smithi078.stdout:481 2023-01-24T10:48:59.766 INFO:tasks.workunit.client.0.smithi078.stdout:482 2023-01-24T10:48:59.769 INFO:tasks.workunit.client.0.smithi078.stdout:483 2023-01-24T10:48:59.772 INFO:tasks.workunit.client.0.smithi078.stdout:484 2023-01-24T10:48:59.775 INFO:tasks.workunit.client.0.smithi078.stdout:485 2023-01-24T10:48:59.777 INFO:tasks.workunit.client.0.smithi078.stdout:486 2023-01-24T10:48:59.781 INFO:tasks.workunit.client.0.smithi078.stdout:487 2023-01-24T10:48:59.783 INFO:tasks.workunit.client.0.smithi078.stdout:488 2023-01-24T10:48:59.786 INFO:tasks.workunit.client.0.smithi078.stdout:489 2023-01-24T10:48:59.787 INFO:tasks.workunit.client.0.smithi078.stdout:490 2023-01-24T10:48:59.789 INFO:tasks.workunit.client.0.smithi078.stdout:491 2023-01-24T10:48:59.790 INFO:tasks.workunit.client.0.smithi078.stdout:492 2023-01-24T10:48:59.793 INFO:tasks.workunit.client.0.smithi078.stdout:493 2023-01-24T10:48:59.794 INFO:tasks.workunit.client.0.smithi078.stdout:494 2023-01-24T10:48:59.795 INFO:tasks.workunit.client.0.smithi078.stdout:495 2023-01-24T10:48:59.796 INFO:tasks.workunit.client.0.smithi078.stdout:496 2023-01-24T10:48:59.797 INFO:tasks.workunit.client.0.smithi078.stdout:497 2023-01-24T10:48:59.800 INFO:tasks.workunit.client.0.smithi078.stdout:498 2023-01-24T10:48:59.803 INFO:tasks.workunit.client.0.smithi078.stdout:499 2023-01-24T10:48:59.806 INFO:tasks.workunit.client.0.smithi078.stdout:500 2023-01-24T10:48:59.808 INFO:tasks.workunit.client.0.smithi078.stdout:501 2023-01-24T10:48:59.811 INFO:tasks.workunit.client.0.smithi078.stdout:502 2023-01-24T10:48:59.813 INFO:tasks.workunit.client.0.smithi078.stdout:503 2023-01-24T10:48:59.816 INFO:tasks.workunit.client.0.smithi078.stdout:504 2023-01-24T10:48:59.819 INFO:tasks.workunit.client.0.smithi078.stdout:505 2023-01-24T10:48:59.821 INFO:tasks.workunit.client.0.smithi078.stdout:506 2023-01-24T10:48:59.824 INFO:tasks.workunit.client.0.smithi078.stdout:507 2023-01-24T10:48:59.827 INFO:tasks.workunit.client.0.smithi078.stdout:508 2023-01-24T10:48:59.828 INFO:tasks.workunit.client.0.smithi078.stdout:509 2023-01-24T10:48:59.830 INFO:tasks.workunit.client.0.smithi078.stdout:510 2023-01-24T10:48:59.830 INFO:tasks.workunit.client.0.smithi078.stdout:511 2023-01-24T10:48:59.831 INFO:tasks.workunit.client.0.smithi078.stdout:512 2023-01-24T10:48:59.832 INFO:tasks.workunit.client.0.smithi078.stdout:513 2023-01-24T10:48:59.833 INFO:tasks.workunit.client.0.smithi078.stdout:514 2023-01-24T10:48:59.834 INFO:tasks.workunit.client.0.smithi078.stdout:515 2023-01-24T10:48:59.835 INFO:tasks.workunit.client.0.smithi078.stdout:516 2023-01-24T10:48:59.836 INFO:tasks.workunit.client.0.smithi078.stdout:517 2023-01-24T10:48:59.837 INFO:tasks.workunit.client.0.smithi078.stdout:518 2023-01-24T10:48:59.838 INFO:tasks.workunit.client.0.smithi078.stdout:519 2023-01-24T10:48:59.840 INFO:tasks.workunit.client.0.smithi078.stdout:520 2023-01-24T10:48:59.841 INFO:tasks.workunit.client.0.smithi078.stdout:521 2023-01-24T10:48:59.844 INFO:tasks.workunit.client.0.smithi078.stdout:522 2023-01-24T10:48:59.846 INFO:tasks.workunit.client.0.smithi078.stdout:523 2023-01-24T10:48:59.850 INFO:tasks.workunit.client.0.smithi078.stdout:524 2023-01-24T10:48:59.851 INFO:tasks.workunit.client.0.smithi078.stdout:525 2023-01-24T10:48:59.854 INFO:tasks.workunit.client.0.smithi078.stdout:526 2023-01-24T10:48:59.857 INFO:tasks.workunit.client.0.smithi078.stdout:527 2023-01-24T10:48:59.859 INFO:tasks.workunit.client.0.smithi078.stdout:528 2023-01-24T10:48:59.861 INFO:tasks.workunit.client.0.smithi078.stdout:529 2023-01-24T10:48:59.864 INFO:tasks.workunit.client.0.smithi078.stdout:530 2023-01-24T10:48:59.866 INFO:tasks.workunit.client.0.smithi078.stdout:531 2023-01-24T10:48:59.868 INFO:tasks.workunit.client.0.smithi078.stdout:532 2023-01-24T10:48:59.871 INFO:tasks.workunit.client.0.smithi078.stdout:533 2023-01-24T10:48:59.872 INFO:tasks.workunit.client.0.smithi078.stdout:534 2023-01-24T10:48:59.874 INFO:tasks.workunit.client.0.smithi078.stdout:535 2023-01-24T10:48:59.875 INFO:tasks.workunit.client.0.smithi078.stdout:536 2023-01-24T10:48:59.876 INFO:tasks.workunit.client.0.smithi078.stdout:537 2023-01-24T10:48:59.877 INFO:tasks.workunit.client.0.smithi078.stdout:538 2023-01-24T10:48:59.878 INFO:tasks.workunit.client.0.smithi078.stdout:539 2023-01-24T10:48:59.879 INFO:tasks.workunit.client.0.smithi078.stdout:540 2023-01-24T10:48:59.880 INFO:tasks.workunit.client.0.smithi078.stdout:541 2023-01-24T10:48:59.882 INFO:tasks.workunit.client.0.smithi078.stdout:542 2023-01-24T10:48:59.883 INFO:tasks.workunit.client.0.smithi078.stdout:543 2023-01-24T10:48:59.884 INFO:tasks.workunit.client.0.smithi078.stdout:544 2023-01-24T10:48:59.885 INFO:tasks.workunit.client.0.smithi078.stdout:545 2023-01-24T10:48:59.886 INFO:tasks.workunit.client.0.smithi078.stdout:546 2023-01-24T10:48:59.887 INFO:tasks.workunit.client.0.smithi078.stdout:547 2023-01-24T10:48:59.889 INFO:tasks.workunit.client.0.smithi078.stdout:548 2023-01-24T10:48:59.890 INFO:tasks.workunit.client.0.smithi078.stdout:549 2023-01-24T10:48:59.891 INFO:tasks.workunit.client.0.smithi078.stdout:550 2023-01-24T10:48:59.893 INFO:tasks.workunit.client.0.smithi078.stdout:551 2023-01-24T10:48:59.897 INFO:tasks.workunit.client.0.smithi078.stdout:552 2023-01-24T10:48:59.898 INFO:tasks.workunit.client.0.smithi078.stdout:553 2023-01-24T10:48:59.901 INFO:tasks.workunit.client.0.smithi078.stdout:554 2023-01-24T10:48:59.904 INFO:tasks.workunit.client.0.smithi078.stdout:555 2023-01-24T10:48:59.905 INFO:tasks.workunit.client.0.smithi078.stdout:556 2023-01-24T10:48:59.907 INFO:tasks.workunit.client.0.smithi078.stdout:557 2023-01-24T10:48:59.909 INFO:tasks.workunit.client.0.smithi078.stdout:558 2023-01-24T10:48:59.912 INFO:tasks.workunit.client.0.smithi078.stdout:559 2023-01-24T10:48:59.915 INFO:tasks.workunit.client.0.smithi078.stdout:560 2023-01-24T10:48:59.917 INFO:tasks.workunit.client.0.smithi078.stdout:561 2023-01-24T10:48:59.919 INFO:tasks.workunit.client.0.smithi078.stdout:562 2023-01-24T10:48:59.921 INFO:tasks.workunit.client.0.smithi078.stdout:563 2023-01-24T10:48:59.922 INFO:tasks.workunit.client.0.smithi078.stdout:564 2023-01-24T10:48:59.923 INFO:tasks.workunit.client.0.smithi078.stdout:565 2023-01-24T10:48:59.924 INFO:tasks.workunit.client.0.smithi078.stdout:566 2023-01-24T10:48:59.925 INFO:tasks.workunit.client.0.smithi078.stdout:567 2023-01-24T10:48:59.926 INFO:tasks.workunit.client.0.smithi078.stdout:568 2023-01-24T10:48:59.927 INFO:tasks.workunit.client.0.smithi078.stdout:569 2023-01-24T10:48:59.928 INFO:tasks.workunit.client.0.smithi078.stdout:570 2023-01-24T10:48:59.929 INFO:tasks.workunit.client.0.smithi078.stdout:571 2023-01-24T10:48:59.930 INFO:tasks.workunit.client.0.smithi078.stdout:572 2023-01-24T10:48:59.931 INFO:tasks.workunit.client.0.smithi078.stdout:573 2023-01-24T10:48:59.932 INFO:tasks.workunit.client.0.smithi078.stdout:574 2023-01-24T10:48:59.934 INFO:tasks.workunit.client.0.smithi078.stdout:575 2023-01-24T10:48:59.938 INFO:tasks.workunit.client.0.smithi078.stdout:576 2023-01-24T10:48:59.938 INFO:tasks.workunit.client.0.smithi078.stdout:577 2023-01-24T10:48:59.938 INFO:tasks.workunit.client.0.smithi078.stdout:578 2023-01-24T10:48:59.938 INFO:tasks.workunit.client.0.smithi078.stdout:579 2023-01-24T10:48:59.939 INFO:tasks.workunit.client.0.smithi078.stdout:580 2023-01-24T10:48:59.940 INFO:tasks.workunit.client.0.smithi078.stdout:581 2023-01-24T10:48:59.941 INFO:tasks.workunit.client.0.smithi078.stdout:582 2023-01-24T10:48:59.942 INFO:tasks.workunit.client.0.smithi078.stdout:583 2023-01-24T10:48:59.943 INFO:tasks.workunit.client.0.smithi078.stdout:584 2023-01-24T10:48:59.944 INFO:tasks.workunit.client.0.smithi078.stdout:585 2023-01-24T10:48:59.945 INFO:tasks.workunit.client.0.smithi078.stdout:586 2023-01-24T10:48:59.946 INFO:tasks.workunit.client.0.smithi078.stdout:587 2023-01-24T10:48:59.949 INFO:tasks.workunit.client.0.smithi078.stdout:588 2023-01-24T10:48:59.952 INFO:tasks.workunit.client.0.smithi078.stdout:589 2023-01-24T10:48:59.955 INFO:tasks.workunit.client.0.smithi078.stdout:590 2023-01-24T10:48:59.957 INFO:tasks.workunit.client.0.smithi078.stdout:591 2023-01-24T10:48:59.960 INFO:tasks.workunit.client.0.smithi078.stdout:592 2023-01-24T10:48:59.964 INFO:tasks.workunit.client.0.smithi078.stdout:593 2023-01-24T10:48:59.966 INFO:tasks.workunit.client.0.smithi078.stdout:594 2023-01-24T10:48:59.969 INFO:tasks.workunit.client.0.smithi078.stdout:595 2023-01-24T10:48:59.972 INFO:tasks.workunit.client.0.smithi078.stdout:596 2023-01-24T10:48:59.975 INFO:tasks.workunit.client.0.smithi078.stdout:597 2023-01-24T10:48:59.977 INFO:tasks.workunit.client.0.smithi078.stdout:598 2023-01-24T10:48:59.980 INFO:tasks.workunit.client.0.smithi078.stdout:599 2023-01-24T10:48:59.983 INFO:tasks.workunit.client.0.smithi078.stdout:600 2023-01-24T10:48:59.984 INFO:tasks.workunit.client.0.smithi078.stdout:601 2023-01-24T10:48:59.985 INFO:tasks.workunit.client.0.smithi078.stdout:602 2023-01-24T10:48:59.986 INFO:tasks.workunit.client.0.smithi078.stdout:603 2023-01-24T10:48:59.987 INFO:tasks.workunit.client.0.smithi078.stdout:604 2023-01-24T10:48:59.988 INFO:tasks.workunit.client.0.smithi078.stdout:605 2023-01-24T10:48:59.989 INFO:tasks.workunit.client.0.smithi078.stdout:606 2023-01-24T10:48:59.990 INFO:tasks.workunit.client.0.smithi078.stdout:607 2023-01-24T10:48:59.992 INFO:tasks.workunit.client.0.smithi078.stdout:608 2023-01-24T10:48:59.993 INFO:tasks.workunit.client.0.smithi078.stdout:609 2023-01-24T10:48:59.996 INFO:tasks.workunit.client.0.smithi078.stdout:610 2023-01-24T10:48:59.998 INFO:tasks.workunit.client.0.smithi078.stdout:611 2023-01-24T10:49:00.001 INFO:tasks.workunit.client.0.smithi078.stdout:612 2023-01-24T10:49:00.005 INFO:tasks.workunit.client.0.smithi078.stdout:613 2023-01-24T10:49:00.008 INFO:tasks.workunit.client.0.smithi078.stdout:614 2023-01-24T10:49:00.009 INFO:tasks.workunit.client.0.smithi078.stdout:615 2023-01-24T10:49:00.011 INFO:tasks.workunit.client.0.smithi078.stdout:616 2023-01-24T10:49:00.014 INFO:tasks.workunit.client.0.smithi078.stdout:617 2023-01-24T10:49:00.017 INFO:tasks.workunit.client.0.smithi078.stdout:618 2023-01-24T10:49:00.018 INFO:tasks.workunit.client.0.smithi078.stdout:619 2023-01-24T10:49:00.020 INFO:tasks.workunit.client.0.smithi078.stdout:620 2023-01-24T10:49:00.022 INFO:tasks.workunit.client.0.smithi078.stdout:621 2023-01-24T10:49:00.024 INFO:tasks.workunit.client.0.smithi078.stdout:622 2023-01-24T10:49:00.026 INFO:tasks.workunit.client.0.smithi078.stdout:623 2023-01-24T10:49:00.029 INFO:tasks.workunit.client.0.smithi078.stdout:624 2023-01-24T10:49:00.032 INFO:tasks.workunit.client.0.smithi078.stdout:625 2023-01-24T10:49:00.034 INFO:tasks.workunit.client.0.smithi078.stdout:626 2023-01-24T10:49:00.037 INFO:tasks.workunit.client.0.smithi078.stdout:627 2023-01-24T10:49:00.038 INFO:tasks.workunit.client.0.smithi078.stdout:628 2023-01-24T10:49:00.040 INFO:tasks.workunit.client.0.smithi078.stdout:629 2023-01-24T10:49:00.043 INFO:tasks.workunit.client.0.smithi078.stdout:630 2023-01-24T10:49:00.045 INFO:tasks.workunit.client.0.smithi078.stdout:631 2023-01-24T10:49:00.047 INFO:tasks.workunit.client.0.smithi078.stdout:632 2023-01-24T10:49:00.048 INFO:tasks.workunit.client.0.smithi078.stdout:633 2023-01-24T10:49:00.049 INFO:tasks.workunit.client.0.smithi078.stdout:634 2023-01-24T10:49:00.052 INFO:tasks.workunit.client.0.smithi078.stdout:635 2023-01-24T10:49:00.053 INFO:tasks.workunit.client.0.smithi078.stdout:636 2023-01-24T10:49:00.054 INFO:tasks.workunit.client.0.smithi078.stdout:637 2023-01-24T10:49:00.056 INFO:tasks.workunit.client.0.smithi078.stdout:638 2023-01-24T10:49:00.057 INFO:tasks.workunit.client.0.smithi078.stdout:639 2023-01-24T10:49:00.058 INFO:tasks.workunit.client.0.smithi078.stdout:640 2023-01-24T10:49:00.059 INFO:tasks.workunit.client.0.smithi078.stdout:641 2023-01-24T10:49:00.060 INFO:tasks.workunit.client.0.smithi078.stdout:642 2023-01-24T10:49:00.061 INFO:tasks.workunit.client.0.smithi078.stdout:643 2023-01-24T10:49:00.063 INFO:tasks.workunit.client.0.smithi078.stdout:644 2023-01-24T10:49:00.064 INFO:tasks.workunit.client.0.smithi078.stdout:645 2023-01-24T10:49:00.065 INFO:tasks.workunit.client.0.smithi078.stdout:646 2023-01-24T10:49:00.066 INFO:tasks.workunit.client.0.smithi078.stdout:647 2023-01-24T10:49:00.068 INFO:tasks.workunit.client.0.smithi078.stdout:648 2023-01-24T10:49:00.069 INFO:tasks.workunit.client.0.smithi078.stdout:649 2023-01-24T10:49:00.070 INFO:tasks.workunit.client.0.smithi078.stdout:650 2023-01-24T10:49:00.072 INFO:tasks.workunit.client.0.smithi078.stdout:651 2023-01-24T10:49:00.073 INFO:tasks.workunit.client.0.smithi078.stdout:652 2023-01-24T10:49:00.075 INFO:tasks.workunit.client.0.smithi078.stdout:653 2023-01-24T10:49:00.076 INFO:tasks.workunit.client.0.smithi078.stdout:654 2023-01-24T10:49:00.078 INFO:tasks.workunit.client.0.smithi078.stdout:655 2023-01-24T10:49:00.079 INFO:tasks.workunit.client.0.smithi078.stdout:656 2023-01-24T10:49:00.080 INFO:tasks.workunit.client.0.smithi078.stdout:657 2023-01-24T10:49:00.081 INFO:tasks.workunit.client.0.smithi078.stdout:658 2023-01-24T10:49:00.082 INFO:tasks.workunit.client.0.smithi078.stdout:659 2023-01-24T10:49:00.083 INFO:tasks.workunit.client.0.smithi078.stdout:660 2023-01-24T10:49:00.084 INFO:tasks.workunit.client.0.smithi078.stdout:661 2023-01-24T10:49:00.085 INFO:tasks.workunit.client.0.smithi078.stdout:662 2023-01-24T10:49:00.087 INFO:tasks.workunit.client.0.smithi078.stdout:663 2023-01-24T10:49:00.088 INFO:tasks.workunit.client.0.smithi078.stdout:664 2023-01-24T10:49:00.089 INFO:tasks.workunit.client.0.smithi078.stdout:665 2023-01-24T10:49:00.090 INFO:tasks.workunit.client.0.smithi078.stdout:666 2023-01-24T10:49:00.093 INFO:tasks.workunit.client.0.smithi078.stdout:667 2023-01-24T10:49:00.095 INFO:tasks.workunit.client.0.smithi078.stdout:668 2023-01-24T10:49:00.098 INFO:tasks.workunit.client.0.smithi078.stdout:669 2023-01-24T10:49:00.101 INFO:tasks.workunit.client.0.smithi078.stdout:670 2023-01-24T10:49:00.103 INFO:tasks.workunit.client.0.smithi078.stdout:671 2023-01-24T10:49:00.104 INFO:tasks.workunit.client.0.smithi078.stdout:672 2023-01-24T10:49:00.105 INFO:tasks.workunit.client.0.smithi078.stdout:673 2023-01-24T10:49:00.108 INFO:tasks.workunit.client.0.smithi078.stdout:674 2023-01-24T10:49:00.110 INFO:tasks.workunit.client.0.smithi078.stdout:675 2023-01-24T10:49:00.113 INFO:tasks.workunit.client.0.smithi078.stdout:676 2023-01-24T10:49:00.116 INFO:tasks.workunit.client.0.smithi078.stdout:677 2023-01-24T10:49:00.117 INFO:tasks.workunit.client.0.smithi078.stdout:678 2023-01-24T10:49:00.120 INFO:tasks.workunit.client.0.smithi078.stdout:679 2023-01-24T10:49:00.122 INFO:tasks.workunit.client.0.smithi078.stdout:680 2023-01-24T10:49:00.125 INFO:tasks.workunit.client.0.smithi078.stdout:681 2023-01-24T10:49:00.128 INFO:tasks.workunit.client.0.smithi078.stdout:682 2023-01-24T10:49:00.131 INFO:tasks.workunit.client.0.smithi078.stdout:683 2023-01-24T10:49:00.133 INFO:tasks.workunit.client.0.smithi078.stdout:684 2023-01-24T10:49:00.135 INFO:tasks.workunit.client.0.smithi078.stdout:685 2023-01-24T10:49:00.138 INFO:tasks.workunit.client.0.smithi078.stdout:686 2023-01-24T10:49:00.140 INFO:tasks.workunit.client.0.smithi078.stdout:687 2023-01-24T10:49:00.143 INFO:tasks.workunit.client.0.smithi078.stdout:688 2023-01-24T10:49:00.144 INFO:tasks.workunit.client.0.smithi078.stdout:689 2023-01-24T10:49:00.145 INFO:tasks.workunit.client.0.smithi078.stdout:690 2023-01-24T10:49:00.147 INFO:tasks.workunit.client.0.smithi078.stdout:691 2023-01-24T10:49:00.148 INFO:tasks.workunit.client.0.smithi078.stdout:692 2023-01-24T10:49:00.151 INFO:tasks.workunit.client.0.smithi078.stdout:693 2023-01-24T10:49:00.151 INFO:tasks.workunit.client.0.smithi078.stdout:694 2023-01-24T10:49:00.152 INFO:tasks.workunit.client.0.smithi078.stdout:695 2023-01-24T10:49:00.159 INFO:tasks.workunit.client.0.smithi078.stdout:696 2023-01-24T10:49:00.161 INFO:tasks.workunit.client.0.smithi078.stdout:697 2023-01-24T10:49:00.161 INFO:tasks.workunit.client.0.smithi078.stdout:698 2023-01-24T10:49:00.161 INFO:tasks.workunit.client.0.smithi078.stdout:699 2023-01-24T10:49:00.161 INFO:tasks.workunit.client.0.smithi078.stdout:700 2023-01-24T10:49:00.162 INFO:tasks.workunit.client.0.smithi078.stdout:701 2023-01-24T10:49:00.162 INFO:tasks.workunit.client.0.smithi078.stdout:702 2023-01-24T10:49:00.162 INFO:tasks.workunit.client.0.smithi078.stdout:703 2023-01-24T10:49:00.163 INFO:tasks.workunit.client.0.smithi078.stdout:704 2023-01-24T10:49:00.164 INFO:tasks.workunit.client.0.smithi078.stdout:705 2023-01-24T10:49:00.165 INFO:tasks.workunit.client.0.smithi078.stdout:706 2023-01-24T10:49:00.166 INFO:tasks.workunit.client.0.smithi078.stdout:707 2023-01-24T10:49:00.168 INFO:tasks.workunit.client.0.smithi078.stdout:708 2023-01-24T10:49:00.169 INFO:tasks.workunit.client.0.smithi078.stdout:709 2023-01-24T10:49:00.170 INFO:tasks.workunit.client.0.smithi078.stdout:710 2023-01-24T10:49:00.171 INFO:tasks.workunit.client.0.smithi078.stdout:711 2023-01-24T10:49:00.173 INFO:tasks.workunit.client.0.smithi078.stdout:712 2023-01-24T10:49:00.174 INFO:tasks.workunit.client.0.smithi078.stdout:713 2023-01-24T10:49:00.175 INFO:tasks.workunit.client.0.smithi078.stdout:714 2023-01-24T10:49:00.176 INFO:tasks.workunit.client.0.smithi078.stdout:715 2023-01-24T10:49:00.180 INFO:tasks.workunit.client.0.smithi078.stdout:716 2023-01-24T10:49:00.181 INFO:tasks.workunit.client.0.smithi078.stdout:717 2023-01-24T10:49:00.181 INFO:tasks.workunit.client.0.smithi078.stdout:718 2023-01-24T10:49:00.182 INFO:tasks.workunit.client.0.smithi078.stdout:719 2023-01-24T10:49:00.182 INFO:tasks.workunit.client.0.smithi078.stdout:720 2023-01-24T10:49:00.183 INFO:tasks.workunit.client.0.smithi078.stdout:721 2023-01-24T10:49:00.185 INFO:tasks.workunit.client.0.smithi078.stdout:722 2023-01-24T10:49:00.186 INFO:tasks.workunit.client.0.smithi078.stdout:723 2023-01-24T10:49:00.187 INFO:tasks.workunit.client.0.smithi078.stdout:724 2023-01-24T10:49:00.188 INFO:tasks.workunit.client.0.smithi078.stdout:725 2023-01-24T10:49:00.190 INFO:tasks.workunit.client.0.smithi078.stdout:726 2023-01-24T10:49:00.191 INFO:tasks.workunit.client.0.smithi078.stdout:727 2023-01-24T10:49:00.192 INFO:tasks.workunit.client.0.smithi078.stdout:728 2023-01-24T10:49:00.193 INFO:tasks.workunit.client.0.smithi078.stdout:729 2023-01-24T10:49:00.194 INFO:tasks.workunit.client.0.smithi078.stdout:730 2023-01-24T10:49:00.195 INFO:tasks.workunit.client.0.smithi078.stdout:731 2023-01-24T10:49:00.197 INFO:tasks.workunit.client.0.smithi078.stdout:732 2023-01-24T10:49:00.198 INFO:tasks.workunit.client.0.smithi078.stdout:733 2023-01-24T10:49:00.199 INFO:tasks.workunit.client.0.smithi078.stdout:734 2023-01-24T10:49:00.200 INFO:tasks.workunit.client.0.smithi078.stdout:735 2023-01-24T10:49:00.201 INFO:tasks.workunit.client.0.smithi078.stdout:736 2023-01-24T10:49:00.203 INFO:tasks.workunit.client.0.smithi078.stdout:737 2023-01-24T10:49:00.204 INFO:tasks.workunit.client.0.smithi078.stdout:738 2023-01-24T10:49:00.205 INFO:tasks.workunit.client.0.smithi078.stdout:739 2023-01-24T10:49:00.208 INFO:tasks.workunit.client.0.smithi078.stdout:740 2023-01-24T10:49:00.211 INFO:tasks.workunit.client.0.smithi078.stdout:741 2023-01-24T10:49:00.213 INFO:tasks.workunit.client.0.smithi078.stdout:742 2023-01-24T10:49:00.214 INFO:tasks.workunit.client.0.smithi078.stdout:743 2023-01-24T10:49:00.217 INFO:tasks.workunit.client.0.smithi078.stdout:744 2023-01-24T10:49:00.220 INFO:tasks.workunit.client.0.smithi078.stdout:745 2023-01-24T10:49:00.223 INFO:tasks.workunit.client.0.smithi078.stdout:746 2023-01-24T10:49:00.227 INFO:tasks.workunit.client.0.smithi078.stdout:747 2023-01-24T10:49:00.228 INFO:tasks.workunit.client.0.smithi078.stdout:748 2023-01-24T10:49:00.229 INFO:tasks.workunit.client.0.smithi078.stdout:749 2023-01-24T10:49:00.231 INFO:tasks.workunit.client.0.smithi078.stdout:750 2023-01-24T10:49:00.234 INFO:tasks.workunit.client.0.smithi078.stdout:751 2023-01-24T10:49:00.237 INFO:tasks.workunit.client.0.smithi078.stdout:752 2023-01-24T10:49:00.240 INFO:tasks.workunit.client.0.smithi078.stdout:753 2023-01-24T10:49:00.242 INFO:tasks.workunit.client.0.smithi078.stdout:754 2023-01-24T10:49:00.245 INFO:tasks.workunit.client.0.smithi078.stdout:755 2023-01-24T10:49:00.246 INFO:tasks.workunit.client.0.smithi078.stdout:756 2023-01-24T10:49:00.249 INFO:tasks.workunit.client.0.smithi078.stdout:757 2023-01-24T10:49:00.251 INFO:tasks.workunit.client.0.smithi078.stdout:758 2023-01-24T10:49:00.253 INFO:tasks.workunit.client.0.smithi078.stdout:759 2023-01-24T10:49:00.256 INFO:tasks.workunit.client.0.smithi078.stdout:760 2023-01-24T10:49:00.259 INFO:tasks.workunit.client.0.smithi078.stdout:761 2023-01-24T10:49:00.262 INFO:tasks.workunit.client.0.smithi078.stdout:762 2023-01-24T10:49:00.264 INFO:tasks.workunit.client.0.smithi078.stdout:763 2023-01-24T10:49:00.270 INFO:tasks.workunit.client.0.smithi078.stdout:764 2023-01-24T10:49:00.271 INFO:tasks.workunit.client.0.smithi078.stdout:765 2023-01-24T10:49:00.274 INFO:tasks.workunit.client.0.smithi078.stdout:766 2023-01-24T10:49:00.277 INFO:tasks.workunit.client.0.smithi078.stdout:767 2023-01-24T10:49:00.279 INFO:tasks.workunit.client.0.smithi078.stdout:768 2023-01-24T10:49:00.282 INFO:tasks.workunit.client.0.smithi078.stdout:769 2023-01-24T10:49:00.285 INFO:tasks.workunit.client.0.smithi078.stdout:770 2023-01-24T10:49:00.286 INFO:tasks.workunit.client.0.smithi078.stdout:771 2023-01-24T10:49:00.288 INFO:tasks.workunit.client.0.smithi078.stdout:772 2023-01-24T10:49:00.289 INFO:tasks.workunit.client.0.smithi078.stdout:773 2023-01-24T10:49:00.292 INFO:tasks.workunit.client.0.smithi078.stdout:774 2023-01-24T10:49:00.295 INFO:tasks.workunit.client.0.smithi078.stdout:775 2023-01-24T10:49:00.297 INFO:tasks.workunit.client.0.smithi078.stdout:776 2023-01-24T10:49:00.300 INFO:tasks.workunit.client.0.smithi078.stdout:777 2023-01-24T10:49:00.303 INFO:tasks.workunit.client.0.smithi078.stdout:778 2023-01-24T10:49:00.304 INFO:tasks.workunit.client.0.smithi078.stdout:779 2023-01-24T10:49:00.307 INFO:tasks.workunit.client.0.smithi078.stdout:780 2023-01-24T10:49:00.309 INFO:tasks.workunit.client.0.smithi078.stdout:781 2023-01-24T10:49:00.312 INFO:tasks.workunit.client.0.smithi078.stdout:782 2023-01-24T10:49:00.315 INFO:tasks.workunit.client.0.smithi078.stdout:783 2023-01-24T10:49:00.317 INFO:tasks.workunit.client.0.smithi078.stdout:784 2023-01-24T10:49:00.320 INFO:tasks.workunit.client.0.smithi078.stdout:785 2023-01-24T10:49:00.321 INFO:tasks.workunit.client.0.smithi078.stdout:786 2023-01-24T10:49:00.324 INFO:tasks.workunit.client.0.smithi078.stdout:787 2023-01-24T10:49:00.325 INFO:tasks.workunit.client.0.smithi078.stdout:788 2023-01-24T10:49:00.326 INFO:tasks.workunit.client.0.smithi078.stdout:789 2023-01-24T10:49:00.326 INFO:tasks.workunit.client.0.smithi078.stdout:790 2023-01-24T10:49:00.327 INFO:tasks.workunit.client.0.smithi078.stdout:791 2023-01-24T10:49:00.328 INFO:tasks.workunit.client.0.smithi078.stdout:792 2023-01-24T10:49:00.329 INFO:tasks.workunit.client.0.smithi078.stdout:793 2023-01-24T10:49:00.331 INFO:tasks.workunit.client.0.smithi078.stdout:794 2023-01-24T10:49:00.332 INFO:tasks.workunit.client.0.smithi078.stdout:795 2023-01-24T10:49:00.333 INFO:tasks.workunit.client.0.smithi078.stdout:796 2023-01-24T10:49:00.335 INFO:tasks.workunit.client.0.smithi078.stdout:797 2023-01-24T10:49:00.336 INFO:tasks.workunit.client.0.smithi078.stdout:798 2023-01-24T10:49:00.337 INFO:tasks.workunit.client.0.smithi078.stdout:799 2023-01-24T10:49:00.338 INFO:tasks.workunit.client.0.smithi078.stdout:800 2023-01-24T10:49:00.339 INFO:tasks.workunit.client.0.smithi078.stdout:801 2023-01-24T10:49:00.341 INFO:tasks.workunit.client.0.smithi078.stdout:802 2023-01-24T10:49:00.342 INFO:tasks.workunit.client.0.smithi078.stdout:803 2023-01-24T10:49:00.343 INFO:tasks.workunit.client.0.smithi078.stdout:804 2023-01-24T10:49:00.344 INFO:tasks.workunit.client.0.smithi078.stdout:805 2023-01-24T10:49:00.345 INFO:tasks.workunit.client.0.smithi078.stdout:806 2023-01-24T10:49:00.347 INFO:tasks.workunit.client.0.smithi078.stdout:807 2023-01-24T10:49:00.348 INFO:tasks.workunit.client.0.smithi078.stdout:808 2023-01-24T10:49:00.350 INFO:tasks.workunit.client.0.smithi078.stdout:809 2023-01-24T10:49:00.351 INFO:tasks.workunit.client.0.smithi078.stdout:810 2023-01-24T10:49:00.352 INFO:tasks.workunit.client.0.smithi078.stdout:811 2023-01-24T10:49:00.354 INFO:tasks.workunit.client.0.smithi078.stdout:812 2023-01-24T10:49:00.357 INFO:tasks.workunit.client.0.smithi078.stdout:813 2023-01-24T10:49:00.360 INFO:tasks.workunit.client.0.smithi078.stdout:814 2023-01-24T10:49:00.366 INFO:tasks.workunit.client.0.smithi078.stdout:815 2023-01-24T10:49:00.366 INFO:tasks.workunit.client.0.smithi078.stdout:816 2023-01-24T10:49:00.368 INFO:tasks.workunit.client.0.smithi078.stdout:817 2023-01-24T10:49:00.370 INFO:tasks.workunit.client.0.smithi078.stdout:818 2023-01-24T10:49:00.373 INFO:tasks.workunit.client.0.smithi078.stdout:819 2023-01-24T10:49:00.374 INFO:tasks.workunit.client.0.smithi078.stdout:820 2023-01-24T10:49:00.376 INFO:tasks.workunit.client.0.smithi078.stdout:821 2023-01-24T10:49:00.379 INFO:tasks.workunit.client.0.smithi078.stdout:822 2023-01-24T10:49:00.382 INFO:tasks.workunit.client.0.smithi078.stdout:823 2023-01-24T10:49:00.385 INFO:tasks.workunit.client.0.smithi078.stdout:824 2023-01-24T10:49:00.387 INFO:tasks.workunit.client.0.smithi078.stdout:825 2023-01-24T10:49:00.390 INFO:tasks.workunit.client.0.smithi078.stdout:826 2023-01-24T10:49:00.393 INFO:tasks.workunit.client.0.smithi078.stdout:827 2023-01-24T10:49:00.395 INFO:tasks.workunit.client.0.smithi078.stdout:828 2023-01-24T10:49:00.396 INFO:tasks.workunit.client.0.smithi078.stdout:829 2023-01-24T10:49:00.398 INFO:tasks.workunit.client.0.smithi078.stdout:830 2023-01-24T10:49:00.399 INFO:tasks.workunit.client.0.smithi078.stdout:831 2023-01-24T10:49:00.400 INFO:tasks.workunit.client.0.smithi078.stdout:832 2023-01-24T10:49:00.402 INFO:tasks.workunit.client.0.smithi078.stdout:833 2023-01-24T10:49:00.403 INFO:tasks.workunit.client.0.smithi078.stdout:834 2023-01-24T10:49:00.404 INFO:tasks.workunit.client.0.smithi078.stdout:835 2023-01-24T10:49:00.406 INFO:tasks.workunit.client.0.smithi078.stdout:836 2023-01-24T10:49:00.408 INFO:tasks.workunit.client.0.smithi078.stdout:837 2023-01-24T10:49:00.413 INFO:tasks.workunit.client.0.smithi078.stdout:838 2023-01-24T10:49:00.413 INFO:tasks.workunit.client.0.smithi078.stdout:839 2023-01-24T10:49:00.414 INFO:tasks.workunit.client.0.smithi078.stdout:840 2023-01-24T10:49:00.414 INFO:tasks.workunit.client.0.smithi078.stdout:841 2023-01-24T10:49:00.415 INFO:tasks.workunit.client.0.smithi078.stdout:842 2023-01-24T10:49:00.416 INFO:tasks.workunit.client.0.smithi078.stdout:843 2023-01-24T10:49:00.420 INFO:tasks.workunit.client.0.smithi078.stdout:844 2023-01-24T10:49:00.420 INFO:tasks.workunit.client.0.smithi078.stdout:845 2023-01-24T10:49:00.423 INFO:tasks.workunit.client.0.smithi078.stdout:846 2023-01-24T10:49:00.424 INFO:tasks.workunit.client.0.smithi078.stdout:847 2023-01-24T10:49:00.425 INFO:tasks.workunit.client.0.smithi078.stdout:848 2023-01-24T10:49:00.427 INFO:tasks.workunit.client.0.smithi078.stdout:849 2023-01-24T10:49:00.428 INFO:tasks.workunit.client.0.smithi078.stdout:850 2023-01-24T10:49:00.429 INFO:tasks.workunit.client.0.smithi078.stdout:851 2023-01-24T10:49:00.431 INFO:tasks.workunit.client.0.smithi078.stdout:852 2023-01-24T10:49:00.432 INFO:tasks.workunit.client.0.smithi078.stdout:853 2023-01-24T10:49:00.433 INFO:tasks.workunit.client.0.smithi078.stdout:854 2023-01-24T10:49:00.435 INFO:tasks.workunit.client.0.smithi078.stdout:855 2023-01-24T10:49:00.436 INFO:tasks.workunit.client.0.smithi078.stdout:856 2023-01-24T10:49:00.438 INFO:tasks.workunit.client.0.smithi078.stdout:857 2023-01-24T10:49:00.439 INFO:tasks.workunit.client.0.smithi078.stdout:858 2023-01-24T10:49:00.441 INFO:tasks.workunit.client.0.smithi078.stdout:859 2023-01-24T10:49:00.442 INFO:tasks.workunit.client.0.smithi078.stdout:860 2023-01-24T10:49:00.443 INFO:tasks.workunit.client.0.smithi078.stdout:861 2023-01-24T10:49:00.445 INFO:tasks.workunit.client.0.smithi078.stdout:862 2023-01-24T10:49:00.446 INFO:tasks.workunit.client.0.smithi078.stdout:863 2023-01-24T10:49:00.447 INFO:tasks.workunit.client.0.smithi078.stdout:864 2023-01-24T10:49:00.449 INFO:tasks.workunit.client.0.smithi078.stdout:865 2023-01-24T10:49:00.450 INFO:tasks.workunit.client.0.smithi078.stdout:866 2023-01-24T10:49:00.451 INFO:tasks.workunit.client.0.smithi078.stdout:867 2023-01-24T10:49:00.452 INFO:tasks.workunit.client.0.smithi078.stdout:868 2023-01-24T10:49:00.454 INFO:tasks.workunit.client.0.smithi078.stdout:869 2023-01-24T10:49:00.455 INFO:tasks.workunit.client.0.smithi078.stdout:870 2023-01-24T10:49:00.456 INFO:tasks.workunit.client.0.smithi078.stdout:871 2023-01-24T10:49:00.457 INFO:tasks.workunit.client.0.smithi078.stdout:872 2023-01-24T10:49:00.458 INFO:tasks.workunit.client.0.smithi078.stdout:873 2023-01-24T10:49:00.459 INFO:tasks.workunit.client.0.smithi078.stdout:874 2023-01-24T10:49:00.460 INFO:tasks.workunit.client.0.smithi078.stdout:875 2023-01-24T10:49:00.462 INFO:tasks.workunit.client.0.smithi078.stdout:876 2023-01-24T10:49:00.463 INFO:tasks.workunit.client.0.smithi078.stdout:877 2023-01-24T10:49:00.464 INFO:tasks.workunit.client.0.smithi078.stdout:878 2023-01-24T10:49:00.465 INFO:tasks.workunit.client.0.smithi078.stdout:879 2023-01-24T10:49:00.468 INFO:tasks.workunit.client.0.smithi078.stdout:880 2023-01-24T10:49:00.471 INFO:tasks.workunit.client.0.smithi078.stdout:881 2023-01-24T10:49:00.473 INFO:tasks.workunit.client.0.smithi078.stdout:882 2023-01-24T10:49:00.477 INFO:tasks.workunit.client.0.smithi078.stdout:883 2023-01-24T10:49:00.480 INFO:tasks.workunit.client.0.smithi078.stdout:884 2023-01-24T10:49:00.482 INFO:tasks.workunit.client.0.smithi078.stdout:885 2023-01-24T10:49:00.483 INFO:tasks.workunit.client.0.smithi078.stdout:886 2023-01-24T10:49:00.486 INFO:tasks.workunit.client.0.smithi078.stdout:887 2023-01-24T10:49:00.488 INFO:tasks.workunit.client.0.smithi078.stdout:888 2023-01-24T10:49:00.491 INFO:tasks.workunit.client.0.smithi078.stdout:889 2023-01-24T10:49:00.493 INFO:tasks.workunit.client.0.smithi078.stdout:890 2023-01-24T10:49:00.495 INFO:tasks.workunit.client.0.smithi078.stdout:891 2023-01-24T10:49:00.498 INFO:tasks.workunit.client.0.smithi078.stdout:892 2023-01-24T10:49:00.500 INFO:tasks.workunit.client.0.smithi078.stdout:893 2023-01-24T10:49:00.503 INFO:tasks.workunit.client.0.smithi078.stdout:894 2023-01-24T10:49:00.505 INFO:tasks.workunit.client.0.smithi078.stdout:895 2023-01-24T10:49:00.509 INFO:tasks.workunit.client.0.smithi078.stdout:896 2023-01-24T10:49:00.510 INFO:tasks.workunit.client.0.smithi078.stdout:897 2023-01-24T10:49:00.512 INFO:tasks.workunit.client.0.smithi078.stdout:898 2023-01-24T10:49:00.515 INFO:tasks.workunit.client.0.smithi078.stdout:899 2023-01-24T10:49:00.516 INFO:tasks.workunit.client.0.smithi078.stdout:900 2023-01-24T10:49:00.520 INFO:tasks.workunit.client.0.smithi078.stdout:901 2023-01-24T10:49:00.522 INFO:tasks.workunit.client.0.smithi078.stdout:902 2023-01-24T10:49:00.523 INFO:tasks.workunit.client.0.smithi078.stdout:903 2023-01-24T10:49:00.524 INFO:tasks.workunit.client.0.smithi078.stdout:904 2023-01-24T10:49:00.525 INFO:tasks.workunit.client.0.smithi078.stdout:905 2023-01-24T10:49:00.526 INFO:tasks.workunit.client.0.smithi078.stdout:906 2023-01-24T10:49:00.527 INFO:tasks.workunit.client.0.smithi078.stdout:907 2023-01-24T10:49:00.529 INFO:tasks.workunit.client.0.smithi078.stdout:908 2023-01-24T10:49:00.530 INFO:tasks.workunit.client.0.smithi078.stdout:909 2023-01-24T10:49:00.531 INFO:tasks.workunit.client.0.smithi078.stdout:910 2023-01-24T10:49:00.532 INFO:tasks.workunit.client.0.smithi078.stdout:911 2023-01-24T10:49:00.533 INFO:tasks.workunit.client.0.smithi078.stdout:912 2023-01-24T10:49:00.535 INFO:tasks.workunit.client.0.smithi078.stdout:913 2023-01-24T10:49:00.536 INFO:tasks.workunit.client.0.smithi078.stdout:914 2023-01-24T10:49:00.537 INFO:tasks.workunit.client.0.smithi078.stdout:915 2023-01-24T10:49:00.538 INFO:tasks.workunit.client.0.smithi078.stdout:916 2023-01-24T10:49:00.540 INFO:tasks.workunit.client.0.smithi078.stdout:917 2023-01-24T10:49:00.541 INFO:tasks.workunit.client.0.smithi078.stdout:918 2023-01-24T10:49:00.543 INFO:tasks.workunit.client.0.smithi078.stdout:919 2023-01-24T10:49:00.544 INFO:tasks.workunit.client.0.smithi078.stdout:920 2023-01-24T10:49:00.546 INFO:tasks.workunit.client.0.smithi078.stdout:921 2023-01-24T10:49:00.548 INFO:tasks.workunit.client.0.smithi078.stdout:922 2023-01-24T10:49:00.551 INFO:tasks.workunit.client.0.smithi078.stdout:923 2023-01-24T10:49:00.554 INFO:tasks.workunit.client.0.smithi078.stdout:924 2023-01-24T10:49:00.556 INFO:tasks.workunit.client.0.smithi078.stdout:925 2023-01-24T10:49:00.557 INFO:tasks.workunit.client.0.smithi078.stdout:926 2023-01-24T10:49:00.558 INFO:tasks.workunit.client.0.smithi078.stdout:927 2023-01-24T10:49:00.559 INFO:tasks.workunit.client.0.smithi078.stdout:928 2023-01-24T10:49:00.560 INFO:tasks.workunit.client.0.smithi078.stdout:929 2023-01-24T10:49:00.561 INFO:tasks.workunit.client.0.smithi078.stdout:930 2023-01-24T10:49:00.563 INFO:tasks.workunit.client.0.smithi078.stdout:931 2023-01-24T10:49:00.564 INFO:tasks.workunit.client.0.smithi078.stdout:932 2023-01-24T10:49:00.565 INFO:tasks.workunit.client.0.smithi078.stdout:933 2023-01-24T10:49:00.566 INFO:tasks.workunit.client.0.smithi078.stdout:934 2023-01-24T10:49:00.568 INFO:tasks.workunit.client.0.smithi078.stdout:935 2023-01-24T10:49:00.569 INFO:tasks.workunit.client.0.smithi078.stdout:936 2023-01-24T10:49:00.570 INFO:tasks.workunit.client.0.smithi078.stdout:937 2023-01-24T10:49:00.572 INFO:tasks.workunit.client.0.smithi078.stdout:938 2023-01-24T10:49:00.573 INFO:tasks.workunit.client.0.smithi078.stdout:939 2023-01-24T10:49:00.575 INFO:tasks.workunit.client.0.smithi078.stdout:940 2023-01-24T10:49:00.578 INFO:tasks.workunit.client.0.smithi078.stdout:941 2023-01-24T10:49:00.579 INFO:tasks.workunit.client.0.smithi078.stdout:942 2023-01-24T10:49:00.580 INFO:tasks.workunit.client.0.smithi078.stdout:943 2023-01-24T10:49:00.582 INFO:tasks.workunit.client.0.smithi078.stdout:944 2023-01-24T10:49:00.583 INFO:tasks.workunit.client.0.smithi078.stdout:945 2023-01-24T10:49:00.584 INFO:tasks.workunit.client.0.smithi078.stdout:946 2023-01-24T10:49:00.586 INFO:tasks.workunit.client.0.smithi078.stdout:947 2023-01-24T10:49:00.587 INFO:tasks.workunit.client.0.smithi078.stdout:948 2023-01-24T10:49:00.588 INFO:tasks.workunit.client.0.smithi078.stdout:949 2023-01-24T10:49:00.590 INFO:tasks.workunit.client.0.smithi078.stdout:950 2023-01-24T10:49:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:951 2023-01-24T10:49:00.592 INFO:tasks.workunit.client.0.smithi078.stdout:952 2023-01-24T10:49:00.593 INFO:tasks.workunit.client.0.smithi078.stdout:953 2023-01-24T10:49:00.595 INFO:tasks.workunit.client.0.smithi078.stdout:954 2023-01-24T10:49:00.596 INFO:tasks.workunit.client.0.smithi078.stdout:955 2023-01-24T10:49:00.597 INFO:tasks.workunit.client.0.smithi078.stdout:956 2023-01-24T10:49:00.598 INFO:tasks.workunit.client.0.smithi078.stdout:957 2023-01-24T10:49:00.600 INFO:tasks.workunit.client.0.smithi078.stdout:958 2023-01-24T10:49:00.601 INFO:tasks.workunit.client.0.smithi078.stdout:959 2023-01-24T10:49:00.603 INFO:tasks.workunit.client.0.smithi078.stdout:960 2023-01-24T10:49:00.609 INFO:tasks.workunit.client.0.smithi078.stdout:961 2023-01-24T10:49:00.611 INFO:tasks.workunit.client.0.smithi078.stdout:962 2023-01-24T10:49:00.614 INFO:tasks.workunit.client.0.smithi078.stdout:963 2023-01-24T10:49:00.615 INFO:tasks.workunit.client.0.smithi078.stdout:964 2023-01-24T10:49:00.618 INFO:tasks.workunit.client.0.smithi078.stdout:965 2023-01-24T10:49:00.621 INFO:tasks.workunit.client.0.smithi078.stdout:966 2023-01-24T10:49:00.624 INFO:tasks.workunit.client.0.smithi078.stdout:967 2023-01-24T10:49:00.631 INFO:tasks.workunit.client.0.smithi078.stdout:968 2023-01-24T10:49:00.632 INFO:tasks.workunit.client.0.smithi078.stdout:969 2023-01-24T10:49:00.632 INFO:tasks.workunit.client.0.smithi078.stdout:970 2023-01-24T10:49:00.634 INFO:tasks.workunit.client.0.smithi078.stdout:971 2023-01-24T10:49:00.635 INFO:tasks.workunit.client.0.smithi078.stdout:972 2023-01-24T10:49:00.637 INFO:tasks.workunit.client.0.smithi078.stdout:973 2023-01-24T10:49:00.639 INFO:tasks.workunit.client.0.smithi078.stdout:974 2023-01-24T10:49:00.640 INFO:tasks.workunit.client.0.smithi078.stdout:975 2023-01-24T10:49:00.642 INFO:tasks.workunit.client.0.smithi078.stdout:976 2023-01-24T10:49:00.644 INFO:tasks.workunit.client.0.smithi078.stdout:977 2023-01-24T10:49:00.645 INFO:tasks.workunit.client.0.smithi078.stdout:978 2023-01-24T10:49:00.648 INFO:tasks.workunit.client.0.smithi078.stdout:979 2023-01-24T10:49:00.649 INFO:tasks.workunit.client.0.smithi078.stdout:980 2023-01-24T10:49:00.652 INFO:tasks.workunit.client.0.smithi078.stdout:981 2023-01-24T10:49:00.655 INFO:tasks.workunit.client.0.smithi078.stdout:982 2023-01-24T10:49:00.655 INFO:tasks.workunit.client.0.smithi078.stdout:983 2023-01-24T10:49:00.658 INFO:tasks.workunit.client.0.smithi078.stdout:984 2023-01-24T10:49:00.660 INFO:tasks.workunit.client.0.smithi078.stdout:985 2023-01-24T10:49:00.662 INFO:tasks.workunit.client.0.smithi078.stdout:986 2023-01-24T10:49:00.662 INFO:tasks.workunit.client.0.smithi078.stdout:987 2023-01-24T10:49:00.664 INFO:tasks.workunit.client.0.smithi078.stdout:988 2023-01-24T10:49:00.665 INFO:tasks.workunit.client.0.smithi078.stdout:989 2023-01-24T10:49:00.666 INFO:tasks.workunit.client.0.smithi078.stdout:990 2023-01-24T10:49:00.667 INFO:tasks.workunit.client.0.smithi078.stdout:991 2023-01-24T10:49:00.669 INFO:tasks.workunit.client.0.smithi078.stdout:992 2023-01-24T10:49:00.670 INFO:tasks.workunit.client.0.smithi078.stdout:993 2023-01-24T10:49:00.671 INFO:tasks.workunit.client.0.smithi078.stdout:994 2023-01-24T10:49:00.673 INFO:tasks.workunit.client.0.smithi078.stdout:995 2023-01-24T10:49:00.674 INFO:tasks.workunit.client.0.smithi078.stdout:996 2023-01-24T10:49:00.675 INFO:tasks.workunit.client.0.smithi078.stdout:997 2023-01-24T10:49:00.677 INFO:tasks.workunit.client.0.smithi078.stdout:998 2023-01-24T10:49:00.678 INFO:tasks.workunit.client.0.smithi078.stdout:999 2023-01-24T10:49:00.977 INFO:tasks.workunit.client.0.smithi078.stdout:00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2023-01-24T10:49:00.977 INFO:tasks.workunit.client.0.smithi078.stdout:00000010 69 73 |is| 2023-01-24T10:49:00.978 INFO:tasks.workunit.client.0.smithi078.stdout:00000012 2023-01-24T10:49:00.978 INFO:tasks.workunit.client.0.smithi078.stdout:00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2023-01-24T10:49:00.978 INFO:tasks.workunit.client.0.smithi078.stdout:00000010 69 73 |is| 2023-01-24T10:49:00.978 INFO:tasks.workunit.client.0.smithi078.stdout:00000012 2023-01-24T10:49:00.978 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsHello.WriteReturnData (3881 ms) 2023-01-24T10:49:00.979 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsHello.Loud 2023-01-24T10:49:03.990 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsHello.Loud (3013 ms) 2023-01-24T10:49:03.990 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsHello.BadMethods 2023-01-24T10:49:06.017 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-01-24T10:49:06.014+0000 7f58f837b700 -1 osd.2 pg_epoch: 126 pg[35.2( v 126'1 (0'0,126'1] local-lis/les=125/126 n=1 ec=125/125 lis/c=125/125 les/c/f=126/126/0 sis=125) [2,4] r=0 lpr=125 crt=126'1 lcod 0'0 mlcod 0'0 active+clean] method hello.bad_reader tried to read object but is not marked RD 2023-01-24T10:49:06.020 INFO:tasks.ceph.osd.2.smithi078.stderr:2023-01-24T10:49:06.017+0000 7f58f4373700 -1 osd.2 pg_epoch: 126 pg[35.2( v 126'2 (0'0,126'2] local-lis/les=125/126 n=1 ec=125/125 lis/c=125/125 les/c/f=126/126/0 sis=125) [2,4] r=0 lpr=125 crt=126'1 lcod 126'1 mlcod 126'1 active+clean] method hello.bad_writer tried to update object but is not marked WR 2023-01-24T10:49:07.023 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsHello.BadMethods (3032 ms) 2023-01-24T10:49:07.023 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsHello.Filter 2023-01-24T10:49:09.982 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsHello.Filter (2959 ms) 2023-01-24T10:49:09.982 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 6 tests from ClsHello (18787 ms total) 2023-01-24T10:49:09.982 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:49:09.982 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:49:09.983 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 6 tests from 1 test suite ran. (18787 ms total) 2023-01-24T10:49:09.983 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 6 tests. 2023-01-24T10:49:09.984 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:49:09.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:49:10.059 INFO:tasks.workunit:Running workunit cls/test_cls_journal.sh... 2023-01-24T10:49:10.059 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_journal.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_journal.sh 2023-01-24T10:49:10.138 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:49:10.138 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 27 tests from 1 test suite. 2023-01-24T10:49:10.138 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:49:10.139 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 27 tests from TestClsJournal 2023-01-24T10:49:11.996 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.Create 2023-01-24T10:49:12.010 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.Create (14 ms) 2023-01-24T10:49:12.011 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.MinimumSet 2023-01-24T10:49:12.016 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.MinimumSet (6 ms) 2023-01-24T10:49:12.017 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.MinimumSetStale 2023-01-24T10:49:12.022 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.MinimumSetStale (6 ms) 2023-01-24T10:49:12.022 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.MinimumSetOrderConstraint 2023-01-24T10:49:12.027 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.MinimumSetOrderConstraint (5 ms) 2023-01-24T10:49:12.028 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ActiveSet 2023-01-24T10:49:12.032 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ActiveSet (4 ms) 2023-01-24T10:49:12.032 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ActiveSetStale 2023-01-24T10:49:12.036 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ActiveSetStale (5 ms) 2023-01-24T10:49:12.036 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.CreateDuplicate 2023-01-24T10:49:12.039 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.CreateDuplicate (3 ms) 2023-01-24T10:49:12.039 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.GetClient 2023-01-24T10:49:12.044 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.GetClient (5 ms) 2023-01-24T10:49:12.044 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientRegister 2023-01-24T10:49:12.048 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientRegister (4 ms) 2023-01-24T10:49:12.048 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientRegisterDuplicate 2023-01-24T10:49:12.056 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientRegisterDuplicate (7 ms) 2023-01-24T10:49:12.056 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientUpdateData 2023-01-24T10:49:12.069 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientUpdateData (14 ms) 2023-01-24T10:49:12.070 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientUpdateState 2023-01-24T10:49:12.075 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientUpdateState (6 ms) 2023-01-24T10:49:12.075 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientUnregister 2023-01-24T10:49:12.079 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientUnregister (4 ms) 2023-01-24T10:49:12.079 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientUnregisterDNE 2023-01-24T10:49:12.084 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientUnregisterDNE (5 ms) 2023-01-24T10:49:12.084 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientUnregisterPruneTags 2023-01-24T10:49:12.272 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientUnregisterPruneTags (188 ms) 2023-01-24T10:49:12.273 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientCommit 2023-01-24T10:49:12.276 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientCommit (3 ms) 2023-01-24T10:49:12.276 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientCommitInvalid 2023-01-24T10:49:12.281 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientCommitInvalid (5 ms) 2023-01-24T10:49:12.281 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientCommitDNE 2023-01-24T10:49:12.282 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientCommitDNE (1 ms) 2023-01-24T10:49:12.282 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.ClientList 2023-01-24T10:49:12.323 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.ClientList (42 ms) 2023-01-24T10:49:12.324 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.GetNextTagTid 2023-01-24T10:49:12.329 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.GetNextTagTid (6 ms) 2023-01-24T10:49:12.329 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.TagCreate 2023-01-24T10:49:12.340 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.TagCreate (10 ms) 2023-01-24T10:49:12.340 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.TagCreatePrunesTags 2023-01-24T10:49:12.346 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.TagCreatePrunesTags (7 ms) 2023-01-24T10:49:12.347 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.TagList 2023-01-24T10:49:12.512 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.TagList (165 ms) 2023-01-24T10:49:12.512 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.GuardAppend 2023-01-24T10:49:12.516 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.GuardAppend (5 ms) 2023-01-24T10:49:12.516 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.GuardAppendDNE 2023-01-24T10:49:12.517 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.GuardAppendDNE (1 ms) 2023-01-24T10:49:12.517 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.GuardAppendOverflow 2023-01-24T10:49:12.521 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.GuardAppendOverflow (4 ms) 2023-01-24T10:49:12.522 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsJournal.Append 2023-01-24T10:49:12.533 INFO:tasks.workunit.client.0.smithi078.stdout:padding is not enabled 2023-01-24T10:49:12.533 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsJournal.Append (12 ms) 2023-01-24T10:49:13.009 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 27 tests from TestClsJournal (537 ms total) 2023-01-24T10:49:13.009 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:49:13.009 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:49:13.009 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 27 tests from 1 test suite ran. (2869 ms total) 2023-01-24T10:49:13.010 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 27 tests. 2023-01-24T10:49:13.010 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:49:13.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:49:13.076 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2023-01-24T10:49:13.077 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2023-01-24T10:49:13.148 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:49:13.149 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 11 tests from 1 test suite. 2023-01-24T10:49:13.149 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:49:13.149 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 11 tests from ClsLock 2023-01-24T10:49:13.149 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestMultiLocking 2023-01-24T10:49:15.046 INFO:tasks.workunit.client.0.smithi078.stdout:lock: mylock 2023-01-24T10:49:15.047 INFO:tasks.workunit.client.0.smithi078.stdout: lock_type: exclusive 2023-01-24T10:49:15.047 INFO:tasks.workunit.client.0.smithi078.stdout: tag: 2023-01-24T10:49:15.047 INFO:tasks.workunit.client.0.smithi078.stdout: lockers: 2023-01-24T10:49:15.047 INFO:tasks.workunit.client.0.smithi078.stdout: client.4653 expiration=2023-01-24T10:51:15.041669+0000 addr=v1:172.21.15.78:0/1212740888 cookie= 2023-01-24T10:49:15.055 INFO:tasks.workunit.client.0.smithi078.stdout:lock: mylock 2023-01-24T10:49:15.055 INFO:tasks.workunit.client.0.smithi078.stdout: lock_type: shared 2023-01-24T10:49:15.055 INFO:tasks.workunit.client.0.smithi078.stdout: tag: 2023-01-24T10:49:15.055 INFO:tasks.workunit.client.0.smithi078.stdout: lockers: 2023-01-24T10:49:15.056 INFO:tasks.workunit.client.0.smithi078.stdout: client.4653 expiration=2023-01-24T10:51:15.050752+0000 addr=v1:172.21.15.78:0/1212740888 cookie= 2023-01-24T10:49:15.056 INFO:tasks.workunit.client.0.smithi078.stdout: client.4659 expiration=0.000000 addr=v1:172.21.15.78:0/198970969 cookie= 2023-01-24T10:49:15.056 INFO:tasks.workunit.client.0.smithi078.stdout:lock: mylock 2023-01-24T10:49:15.057 INFO:tasks.workunit.client.0.smithi078.stdout: lock_type: shared 2023-01-24T10:49:15.057 INFO:tasks.workunit.client.0.smithi078.stdout: tag: 2023-01-24T10:49:15.057 INFO:tasks.workunit.client.0.smithi078.stdout: lockers: 2023-01-24T10:49:15.057 INFO:tasks.workunit.client.0.smithi078.stdout: client.4659 expiration=0.000000 addr=v1:172.21.15.78:0/198970969 cookie= 2023-01-24T10:49:16.024 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestMultiLocking (2876 ms) 2023-01-24T10:49:16.025 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestMeta 2023-01-24T10:49:18.209 INFO:tasks.workunit.client.0.smithi078.stdout:lock: mylock 2023-01-24T10:49:18.209 INFO:tasks.workunit.client.0.smithi078.stdout: lock_type: shared 2023-01-24T10:49:18.210 INFO:tasks.workunit.client.0.smithi078.stdout: tag: 2023-01-24T10:49:18.210 INFO:tasks.workunit.client.0.smithi078.stdout: lockers: 2023-01-24T10:49:18.210 INFO:tasks.workunit.client.0.smithi078.stdout: client.4526 expiration=0.000000 addr=v1:172.21.15.78:0/285789097 cookie= 2023-01-24T10:49:18.212 INFO:tasks.workunit.client.0.smithi078.stdout:lock: mylock 2023-01-24T10:49:18.213 INFO:tasks.workunit.client.0.smithi078.stdout: lock_type: exclusive 2023-01-24T10:49:18.213 INFO:tasks.workunit.client.0.smithi078.stdout: tag: new_tag 2023-01-24T10:49:18.213 INFO:tasks.workunit.client.0.smithi078.stdout: lockers: 2023-01-24T10:49:18.214 INFO:tasks.workunit.client.0.smithi078.stdout: client.4520 expiration=0.000000 addr=v1:172.21.15.78:0/4042824100 cookie= 2023-01-24T10:49:18.979 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestMeta (2954 ms) 2023-01-24T10:49:18.979 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestCookie 2023-01-24T10:49:21.018 INFO:tasks.workunit.client.0.smithi078.stdout:lock: mylock 2023-01-24T10:49:21.018 INFO:tasks.workunit.client.0.smithi078.stdout: lock_type: exclusive 2023-01-24T10:49:21.018 INFO:tasks.workunit.client.0.smithi078.stdout: tag: 2023-01-24T10:49:21.018 INFO:tasks.workunit.client.0.smithi078.stdout: lockers: 2023-01-24T10:49:21.023 INFO:tasks.workunit.client.0.smithi078.stdout:lock: mylock 2023-01-24T10:49:21.023 INFO:tasks.workunit.client.0.smithi078.stdout: lock_type: shared 2023-01-24T10:49:21.023 INFO:tasks.workunit.client.0.smithi078.stdout: tag: 2023-01-24T10:49:21.023 INFO:tasks.workunit.client.0.smithi078.stdout: lockers: 2023-01-24T10:49:21.024 INFO:tasks.workunit.client.0.smithi078.stdout: client.4677 expiration=0.000000 addr=v1:172.21.15.78:0/2308826410 cookie= 2023-01-24T10:49:21.024 INFO:tasks.workunit.client.0.smithi078.stdout: client.4677 expiration=0.000000 addr=v1:172.21.15.78:0/2308826410 cookie=new cookie 2023-01-24T10:49:21.995 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestCookie (3017 ms) 2023-01-24T10:49:21.996 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestMultipleLocks 2023-01-24T10:49:25.015 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestMultipleLocks (3019 ms) 2023-01-24T10:49:25.015 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestLockDuration 2023-01-24T10:49:32.985 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestLockDuration (7971 ms) 2023-01-24T10:49:32.985 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestAssertLocked 2023-01-24T10:49:36.017 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestAssertLocked (3031 ms) 2023-01-24T10:49:36.017 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestSetCookie 2023-01-24T10:49:38.983 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestSetCookie (2967 ms) 2023-01-24T10:49:38.984 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestRenew 2023-01-24T10:49:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestRenew (21012 ms) 2023-01-24T10:49:59.999 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2023-01-24T10:50:06.426 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (6430 ms) 2023-01-24T10:50:06.426 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2023-01-24T10:50:13.984 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (7558 ms) 2023-01-24T10:50:13.984 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2023-01-24T10:50:21.420 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (7436 ms) 2023-01-24T10:50:21.420 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 11 tests from ClsLock (68271 ms total) 2023-01-24T10:50:21.420 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:50:21.421 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:50:21.421 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 11 tests from 1 test suite ran. (68271 ms total) 2023-01-24T10:50:21.421 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 11 tests. 2023-01-24T10:50:21.422 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:50:21.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:50:21.510 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2023-01-24T10:50:21.511 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2023-01-24T10:50:21.546 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:50:21.546 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 4 tests from 1 test suite. 2023-01-24T10:50:21.546 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:50:21.547 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 4 tests from cls_log 2023-01-24T10:50:21.547 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_log.test_log_add_same_time 2023-01-24T10:50:24.445 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_log.test_log_add_same_time (2899 ms) 2023-01-24T10:50:24.445 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_log.test_log_add_different_time 2023-01-24T10:50:27.464 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_log.test_log_add_different_time (3016 ms) 2023-01-24T10:50:27.464 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_log.trim_by_time 2023-01-24T10:50:29.999 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_log.trim_by_time (2538 ms) 2023-01-24T10:50:29.999 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_log.trim_by_marker 2023-01-24T10:50:33.008 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_log.trim_by_marker (3010 ms) 2023-01-24T10:50:33.009 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 4 tests from cls_log (11463 ms total) 2023-01-24T10:50:33.009 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:50:33.009 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:50:33.009 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 4 tests from 1 test suite ran. (11463 ms total) 2023-01-24T10:50:33.009 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 4 tests. 2023-01-24T10:50:33.011 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:50:33.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:50:33.085 INFO:tasks.workunit:Running workunit cls/test_cls_numops.sh... 2023-01-24T10:50:33.085 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_numops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_numops.sh 2023-01-24T10:50:33.164 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:50:33.165 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 4 tests from 1 test suite. 2023-01-24T10:50:33.165 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:50:33.165 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 4 tests from ClsNumOps 2023-01-24T10:50:33.165 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsNumOps.Add 2023-01-24T10:50:36.044 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsNumOps.Add (2880 ms) 2023-01-24T10:50:36.044 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsNumOps.Sub 2023-01-24T10:50:38.994 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsNumOps.Sub (2949 ms) 2023-01-24T10:50:38.994 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsNumOps.Mul 2023-01-24T10:50:42.011 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsNumOps.Mul (3018 ms) 2023-01-24T10:50:42.011 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] ClsNumOps.Div 2023-01-24T10:50:45.034 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] ClsNumOps.Div (3023 ms) 2023-01-24T10:50:45.035 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 4 tests from ClsNumOps (11870 ms total) 2023-01-24T10:50:45.035 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:50:45.035 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:50:45.035 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 4 tests from 1 test suite ran. (11870 ms total) 2023-01-24T10:50:45.035 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 4 tests. 2023-01-24T10:50:45.036 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:50:45.037 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:50:45.107 INFO:tasks.workunit:Running workunit cls/test_cls_rbd.sh... 2023-01-24T10:50:45.108 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_rbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rbd.sh 2023-01-24T10:50:45.183 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:50:45.184 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 69 tests from 1 test suite. 2023-01-24T10:50:45.184 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:50:45.184 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 69 tests from TestClsRbd 2023-01-24T10:50:47.045 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_all_features 2023-01-24T10:50:47.059 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_all_features (13 ms) 2023-01-24T10:50:47.059 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.copyup 2023-01-24T10:50:47.225 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.copyup (167 ms) 2023-01-24T10:50:47.226 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.sparse_copyup 2023-01-24T10:50:47.252 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.sparse_copyup (27 ms) 2023-01-24T10:50:47.252 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_and_set_id 2023-01-24T10:50:47.262 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_and_set_id (10 ms) 2023-01-24T10:50:47.262 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.add_remove_child 2023-01-24T10:50:47.277 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.add_remove_child (14 ms) 2023-01-24T10:50:47.277 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.directory_methods 2023-01-24T10:50:47.334 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.directory_methods (57 ms) 2023-01-24T10:50:47.334 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.create 2023-01-24T10:50:47.355 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.create (22 ms) 2023-01-24T10:50:47.355 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_features 2023-01-24T10:50:47.357 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_features (2 ms) 2023-01-24T10:50:47.357 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_object_prefix 2023-01-24T10:50:47.359 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_object_prefix (2 ms) 2023-01-24T10:50:47.359 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_create_timestamp 2023-01-24T10:50:47.360 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_create_timestamp (1 ms) 2023-01-24T10:50:47.360 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_access_timestamp 2023-01-24T10:50:47.363 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_access_timestamp (3 ms) 2023-01-24T10:50:47.363 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_modify_timestamp 2023-01-24T10:50:47.365 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_modify_timestamp (2 ms) 2023-01-24T10:50:47.365 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_data_pool 2023-01-24T10:50:47.371 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_data_pool (6 ms) 2023-01-24T10:50:47.371 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.get_size 2023-01-24T10:50:47.377 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.get_size (6 ms) 2023-01-24T10:50:47.378 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.set_size 2023-01-24T10:50:47.384 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.set_size (5 ms) 2023-01-24T10:50:47.384 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.protection_status 2023-01-24T10:50:47.420 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.protection_status (37 ms) 2023-01-24T10:50:47.420 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.snapshot_limits 2023-01-24T10:50:47.427 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.snapshot_limits (8 ms) 2023-01-24T10:50:47.427 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.parents_v1 2023-01-24T10:50:47.478 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.parents_v1 (51 ms) 2023-01-24T10:50:47.479 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.parents_v2 2023-01-24T10:50:47.532 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.parents_v2 (54 ms) 2023-01-24T10:50:47.533 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.snapshots 2023-01-24T10:50:47.570 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.snapshots (38 ms) 2023-01-24T10:50:47.570 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.snapid_race 2023-01-24T10:50:47.580 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.snapid_race (10 ms) 2023-01-24T10:50:47.581 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.stripingv2 2023-01-24T10:50:47.597 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.stripingv2 (16 ms) 2023-01-24T10:50:47.597 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.object_map_save 2023-01-24T10:50:47.602 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.object_map_save (6 ms) 2023-01-24T10:50:47.602 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.object_map_resize 2023-01-24T10:50:47.610 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.object_map_resize (8 ms) 2023-01-24T10:50:47.611 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.object_map_update 2023-01-24T10:50:47.619 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.object_map_update (9 ms) 2023-01-24T10:50:47.620 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.object_map_load_enoent 2023-01-24T10:50:47.620 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.object_map_load_enoent (0 ms) 2023-01-24T10:50:47.620 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.object_map_snap_add 2023-01-24T10:50:47.629 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.object_map_snap_add (9 ms) 2023-01-24T10:50:47.629 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.object_map_snap_remove 2023-01-24T10:50:47.634 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.object_map_snap_remove (6 ms) 2023-01-24T10:50:47.634 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.flags 2023-01-24T10:50:47.639 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.flags (5 ms) 2023-01-24T10:50:47.639 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.metadata 2023-01-24T10:50:47.665 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.metadata (25 ms) 2023-01-24T10:50:47.666 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.set_features 2023-01-24T10:50:47.673 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.set_features (9 ms) 2023-01-24T10:50:47.674 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.mirror 2023-01-24T10:50:47.717 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.mirror (43 ms) 2023-01-24T10:50:47.717 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.mirror_image 2023-01-24T10:50:47.738 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.mirror_image (22 ms) 2023-01-24T10:50:47.738 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.mirror_image_status 2023-01-24T10:50:51.401 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.mirror_image_status (3662 ms) 2023-01-24T10:50:51.401 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.mirror_image_map 2023-01-24T10:50:51.501 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.mirror_image_map (100 ms) 2023-01-24T10:50:51.501 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.mirror_instances 2023-01-24T10:50:51.517 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.mirror_instances (17 ms) 2023-01-24T10:50:51.518 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.mirror_snapshot 2023-01-24T10:50:51.538 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.mirror_snapshot (20 ms) 2023-01-24T10:50:51.538 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_dir_list 2023-01-24T10:50:51.540 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_dir_list (3 ms) 2023-01-24T10:50:51.540 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_dir_add 2023-01-24T10:50:51.544 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_dir_add (4 ms) 2023-01-24T10:50:51.544 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.dir_add_already_existing 2023-01-24T10:50:51.549 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.dir_add_already_existing (5 ms) 2023-01-24T10:50:51.550 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_dir_rename 2023-01-24T10:50:51.562 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_dir_rename (13 ms) 2023-01-24T10:50:51.563 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_dir_remove 2023-01-24T10:50:51.566 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_dir_remove (3 ms) 2023-01-24T10:50:51.566 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_dir_remove_missing 2023-01-24T10:50:51.572 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_dir_remove_missing (6 ms) 2023-01-24T10:50:51.572 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_image_add 2023-01-24T10:50:51.574 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_image_add (3 ms) 2023-01-24T10:50:51.574 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_image_remove 2023-01-24T10:50:51.578 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_image_remove (4 ms) 2023-01-24T10:50:51.579 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_image_list 2023-01-24T10:50:51.581 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_image_list (3 ms) 2023-01-24T10:50:51.581 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_image_clean 2023-01-24T10:50:51.587 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_image_clean (5 ms) 2023-01-24T10:50:51.587 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.image_group_add 2023-01-24T10:50:51.589 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.image_group_add (3 ms) 2023-01-24T10:50:51.589 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.image_group_remove 2023-01-24T10:50:51.594 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.image_group_remove (4 ms) 2023-01-24T10:50:51.594 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.image_group_get 2023-01-24T10:50:51.596 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.image_group_get (3 ms) 2023-01-24T10:50:51.596 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_name 2023-01-24T10:50:51.598 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_set_empty_name (2 ms) 2023-01-24T10:50:51.598 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_set_empty_id 2023-01-24T10:50:51.600 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_set_empty_id (2 ms) 2023-01-24T10:50:51.600 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_id 2023-01-24T10:50:51.602 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_id (2 ms) 2023-01-24T10:50:51.603 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_set_duplicate_name 2023-01-24T10:50:51.606 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_set_duplicate_name (4 ms) 2023-01-24T10:50:51.606 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_set 2023-01-24T10:50:51.608 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_set (2 ms) 2023-01-24T10:50:51.609 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_list 2023-01-24T10:50:51.612 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_list (4 ms) 2023-01-24T10:50:51.612 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_list_max_return 2023-01-24T10:50:51.628 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_list_max_return (16 ms) 2023-01-24T10:50:51.628 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_remove 2023-01-24T10:50:51.632 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_remove (4 ms) 2023-01-24T10:50:51.632 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.group_snap_get_by_id 2023-01-24T10:50:51.634 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.group_snap_get_by_id (2 ms) 2023-01-24T10:50:51.634 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.trash_methods 2023-01-24T10:50:51.642 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.trash_methods (7 ms) 2023-01-24T10:50:51.642 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.op_features 2023-01-24T10:50:51.649 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.op_features (8 ms) 2023-01-24T10:50:51.649 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.clone_parent 2023-01-24T10:50:51.683 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.clone_parent (33 ms) 2023-01-24T10:50:51.683 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.clone_parent_ns 2023-01-24T10:50:51.699 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.clone_parent_ns (16 ms) 2023-01-24T10:50:51.699 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.clone_child 2023-01-24T10:50:51.720 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.clone_child (22 ms) 2023-01-24T10:50:51.720 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.namespace_methods 2023-01-24T10:50:51.728 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.namespace_methods (8 ms) 2023-01-24T10:50:51.729 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.migration 2023-01-24T10:50:51.739 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.migration (11 ms) 2023-01-24T10:50:51.739 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.migration_v1 2023-01-24T10:50:51.756 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.migration_v1 (17 ms) 2023-01-24T10:50:51.757 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.assert_snapc_seq 2023-01-24T10:50:52.989 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.assert_snapc_seq (1232 ms) 2023-01-24T10:50:52.989 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] TestClsRbd.sparsify 2023-01-24T10:50:53.064 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] TestClsRbd.sparsify (76 ms) 2023-01-24T10:50:54.000 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 69 tests from TestClsRbd (6019 ms total) 2023-01-24T10:50:54.000 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:50:54.001 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:50:54.001 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 69 tests from 1 test suite ran. (8816 ms total) 2023-01-24T10:50:54.001 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 69 tests. 2023-01-24T10:50:54.002 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:50:54.002 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:50:54.075 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2023-01-24T10:50:54.076 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2023-01-24T10:50:54.153 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:50:54.154 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 5 tests from 1 test suite. 2023-01-24T10:50:54.154 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:50:54.154 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 5 tests from cls_rgw 2023-01-24T10:50:54.154 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.test_implicit 2023-01-24T10:50:57.023 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.test_implicit (2870 ms) 2023-01-24T10:50:57.024 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2023-01-24T10:51:00.037 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3014 ms) 2023-01-24T10:51:00.038 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.test_put_snap 2023-01-24T10:51:07.996 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.test_put_snap (7958 ms) 2023-01-24T10:51:07.996 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.test_explicit 2023-01-24T10:51:11.023 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.test_explicit (3028 ms) 2023-01-24T10:51:11.024 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.set 2023-01-24T10:51:14.040 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.set (3017 ms) 2023-01-24T10:51:14.041 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 5 tests from cls_rgw (19887 ms total) 2023-01-24T10:51:14.041 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:51:14.041 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:51:14.041 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 5 tests from 1 test suite ran. (19887 ms total) 2023-01-24T10:51:14.042 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 5 tests. 2023-01-24T10:51:14.043 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:51:14.043 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:51:14.113 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2023-01-24T10:51:14.114 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2023-01-24T10:51:14.189 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:51:14.189 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 15 tests from 1 test suite. 2023-01-24T10:51:14.190 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:51:14.190 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 15 tests from cls_rgw 2023-01-24T10:51:16.054 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.index_basic 2023-01-24T10:51:16.328 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.index_basic (273 ms) 2023-01-24T10:51:16.328 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2023-01-24T10:51:16.377 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (50 ms) 2023-01-24T10:51:16.378 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.index_remove_object 2023-01-24T10:51:16.430 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.index_remove_object (53 ms) 2023-01-24T10:51:16.430 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.index_suggest 2023-01-24T10:51:18.070 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.index_suggest (1640 ms) 2023-01-24T10:51:18.071 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.index_suggest_complete 2023-01-24T10:51:18.082 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.index_suggest_complete (12 ms) 2023-01-24T10:51:18.083 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.index_list 2023-01-24T10:51:18.113 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.index_list (31 ms) 2023-01-24T10:51:18.114 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.index_list_delimited 2023-01-24T10:52:09.101 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.index_list_delimited (50987 ms) 2023-01-24T10:52:09.101 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.bi_list 2023-01-24T10:52:09.189 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.bi_list (89 ms) 2023-01-24T10:52:09.190 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.gc_set 2023-01-24T10:52:09.220 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.gc_set (31 ms) 2023-01-24T10:52:09.221 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.gc_list 2023-01-24T10:52:09.451 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.gc_list (231 ms) 2023-01-24T10:52:09.452 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.gc_defer 2023-01-24T10:52:17.016 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.gc_defer (7563 ms) 2023-01-24T10:52:17.017 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.usage_basic 2023-01-24T10:52:17.093 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.usage_basic (79 ms) 2023-01-24T10:52:17.093 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2023-01-24T10:52:17.095 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.usage_clear_no_obj (2 ms) 2023-01-24T10:52:17.095 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.usage_clear 2023-01-24T10:52:17.203 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.usage_clear (108 ms) 2023-01-24T10:52:17.204 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw.bi_log_trim 2023-01-24T10:52:17.288 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw.bi_log_trim (85 ms) 2023-01-24T10:52:18.011 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 15 tests from cls_rgw (61234 ms total) 2023-01-24T10:52:18.011 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:52:18.011 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:52:18.012 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 15 tests from 1 test suite ran. (63822 ms total) 2023-01-24T10:52:18.012 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 15 tests. 2023-01-24T10:52:18.013 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:52:18.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:52:18.083 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_gc.sh... 2023-01-24T10:52:18.084 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_rgw_gc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_gc.sh 2023-01-24T10:52:18.158 INFO:tasks.workunit.client.0.smithi078.stdout:Running main() from gmock_main.cc 2023-01-24T10:52:18.158 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 9 tests from 1 test suite. 2023-01-24T10:52:18.159 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:52:18.159 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 9 tests from cls_rgw_gc 2023-01-24T10:52:18.159 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.init 2023-01-24T10:52:20.043 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.init (1885 ms) 2023-01-24T10:52:20.043 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops1 2023-01-24T10:52:20.053 INFO:tasks.workunit.client.0.smithi078.stderr:[ ] list info tag = chain-0 2023-01-24T10:52:20.053 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.gc_queue_ops1 (10 ms) 2023-01-24T10:52:20.053 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops2 2023-01-24T10:52:20.063 INFO:tasks.workunit.client.0.smithi078.stderr:[ ] next_marker is: = 0/4263 2023-01-24T10:52:20.064 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.gc_queue_ops2 (11 ms) 2023-01-24T10:52:20.064 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops5 2023-01-24T10:52:20.073 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.gc_queue_ops5 (10 ms) 2023-01-24T10:52:20.073 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops6 2023-01-24T10:52:20.084 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.gc_queue_ops6 (11 ms) 2023-01-24T10:52:20.084 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops7 2023-01-24T10:52:20.094 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.gc_queue_ops7 (10 ms) 2023-01-24T10:52:20.095 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops8 2023-01-24T10:52:20.105 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.gc_queue_ops8 (11 ms) 2023-01-24T10:52:20.105 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.gc_queue_ops9 2023-01-24T10:52:20.113 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.gc_queue_ops9 (8 ms) 2023-01-24T10:52:20.114 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_gc.finalize 2023-01-24T10:52:21.029 INFO:tasks.workunit.client.0.smithi078.stdout:[ OK ] cls_rgw_gc.finalize (916 ms) 2023-01-24T10:52:21.030 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 9 tests from cls_rgw_gc (2872 ms total) 2023-01-24T10:52:21.030 INFO:tasks.workunit.client.0.smithi078.stdout: 2023-01-24T10:52:21.030 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment tear-down 2023-01-24T10:52:21.030 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] 9 tests from 1 test suite ran. (2872 ms total) 2023-01-24T10:52:21.031 INFO:tasks.workunit.client.0.smithi078.stdout:[ PASSED ] 9 tests. 2023-01-24T10:52:21.032 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T10:52:21.032 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T10:52:21.105 INFO:tasks.workunit:Running workunit cls/test_cls_rgw_stats.sh... 2023-01-24T10:52:21.106 DEBUG:teuthology.orchestra.run.smithi078:workunit test cls/test_cls_rgw_stats.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1a4043421bb637c432ee02d001db3c6ec52c48e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw_stats.sh 2023-01-24T10:52:21.191 INFO:tasks.workunit.client.0.smithi078.stdout:[==========] Running 1 test from 1 test suite. 2023-01-24T10:52:21.192 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] Global test environment set-up. 2023-01-24T10:52:23.051 INFO:tasks.workunit.client.0.smithi078.stdout:[----------] 1 test from cls_rgw_stats 2023-01-24T10:52:23.051 INFO:tasks.workunit.client.0.smithi078.stdout:[ RUN ] cls_rgw_stats.simulate 2023-01-24T10:52:23.068 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.067+0000 7fbe187898c0 1 > prepared operation key=FDZVWZ0V33EB tag=BDCOORCIXU0T type=del size=1959573 2023-01-24T10:52:23.072 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.071+0000 7fbe187898c0 1 > prepared operation key=Z5DPD9MNHN2H tag=PE9JY7785YPS type=del size=3954684 2023-01-24T10:52:23.075 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.073+0000 7fbe187898c0 1 > prepared operation key=Z5DPD9MNHN2H tag=OA3IIZTN25G6 type=del size=2796381 2023-01-24T10:52:23.079 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.077+0000 7fbe187898c0 1 > prepared part key=_multipart_63PDRJCRVEI1.2~D9XZFDI70FFH.0 size=1048576 2023-01-24T10:52:23.080 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.079+0000 7fbe187898c0 1 > prepared part key=_multipart_63PDRJCRVEI1.2~D9XZFDI70FFH.1 size=241080 2023-01-24T10:52:23.080 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.079+0000 7fbe187898c0 1 > prepared multipart upload key=63PDRJCRVEI1 upload=D9XZFDI70FFH tag=N2HK71FF22LZ type=write size=2338232 2023-01-24T10:52:23.083 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.082+0000 7fbe187898c0 1 > prepared operation key=D0CGZ9OQK1AM tag=EILE8277S2O1 type=del size=2427147 2023-01-24T10:52:23.087 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.086+0000 7fbe187898c0 1 > prepared part key=_multipart_ZMEBN0OQ1IPJ.2~U5DE9INXCWXJ.0 size=1048576 2023-01-24T10:52:23.088 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.087+0000 7fbe187898c0 1 > prepared part key=_multipart_ZMEBN0OQ1IPJ.2~U5DE9INXCWXJ.1 size=273500 2023-01-24T10:52:23.089 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.087+0000 7fbe187898c0 1 > prepared multipart upload key=ZMEBN0OQ1IPJ upload=U5DE9INXCWXJ tag=79A9ONP946CG type=write size=2370652 2023-01-24T10:52:23.090 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.088+0000 7fbe187898c0 1 > prepared operation key=YTEKYBINDI7I tag=O5D9N83DHYOG type=del size=1252018 2023-01-24T10:52:23.091 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.090+0000 7fbe187898c0 1 > prepared operation key=3GJVNT6TXTK6 tag=V7U49Y3B8F52 type=del size=2083500 2023-01-24T10:52:23.094 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.092+0000 7fbe187898c0 1 > prepared part key=_multipart_63PDRJCRVEI1.2~C6LGNGEUIXUW.0 size=1048576 2023-01-24T10:52:23.095 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.094+0000 7fbe187898c0 1 > prepared part key=_multipart_63PDRJCRVEI1.2~C6LGNGEUIXUW.1 size=90247 2023-01-24T10:52:23.095 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.094+0000 7fbe187898c0 1 > prepared multipart upload key=63PDRJCRVEI1 upload=C6LGNGEUIXUW tag=JINY1AXUP43W type=write size=2187399 2023-01-24T10:52:23.096 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.095+0000 7fbe187898c0 1 > prepared operation key=TIE8E7PAI3NC tag=GIB5NBLCZZE0 type=del size=1034183 2023-01-24T10:52:23.101 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.097+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~GBK80ON6BP1G.0 size=1048576 2023-01-24T10:52:23.101 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.098+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~GBK80ON6BP1G.1 size=1048576 2023-01-24T10:52:23.101 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.099+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~GBK80ON6BP1G.2 size=960413 2023-01-24T10:52:23.101 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.099+0000 7fbe187898c0 1 > prepared multipart upload key=FDZVWZ0V33EB upload=GBK80ON6BP1G tag=V6WCCP1H8I0V type=write size=4106141 2023-01-24T10:52:23.103 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.102+0000 7fbe187898c0 1 > prepared part key=_multipart_1ZN0Y6AS7ISA.2~RUVJ42PFK77Y.0 size=734331 2023-01-24T10:52:23.103 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.102+0000 7fbe187898c0 1 > prepared multipart upload key=1ZN0Y6AS7ISA upload=RUVJ42PFK77Y tag=QTOVRDRHJ937 type=write size=1782907 2023-01-24T10:52:23.105 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.104+0000 7fbe187898c0 1 > prepared part key=_multipart_S7XZ2Q32GCMK.2~X5SNF3MK1DLZ.0 size=1048576 2023-01-24T10:52:23.106 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.105+0000 7fbe187898c0 1 > prepared part key=_multipart_S7XZ2Q32GCMK.2~X5SNF3MK1DLZ.1 size=852693 2023-01-24T10:52:23.106 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.105+0000 7fbe187898c0 1 > prepared multipart upload key=S7XZ2Q32GCMK upload=X5SNF3MK1DLZ tag=UFJNN0TTVUC0 type=write size=2949845 2023-01-24T10:52:23.108 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.107+0000 7fbe187898c0 1 > prepared operation key=D0CGZ9OQK1AM tag=XU7X1I6K24O3 type=del size=241441 2023-01-24T10:52:23.110 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.108+0000 7fbe187898c0 1 > prepared operation key=1ZN0Y6AS7ISA tag=2OS4C2E8L350 type=write size=943111 2023-01-24T10:52:23.111 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.110+0000 7fbe187898c0 1 > prepared operation key=C9J0MO49LDCS tag=4ERXAXKR7YNI type=del size=54315 2023-01-24T10:52:23.114 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.113+0000 7fbe187898c0 1 < suggested remove operation key=FDZVWZ0V33EB tag=BDCOORCIXU0T type=del 2023-01-24T10:52:23.115 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.114+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=I38MSTVKAOHU type=del size=2294802 2023-01-24T10:52:23.117 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.115+0000 7fbe187898c0 1 < completed delete operation key=Z5DPD9MNHN2H tag=PE9JY7785YPS type=del 2023-01-24T10:52:23.118 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.117+0000 7fbe187898c0 1 > prepared operation key=Z4E99NU2FUTH tag=2RGMZRB5E7WK type=write size=173149 2023-01-24T10:52:23.119 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.118+0000 7fbe187898c0 1 < completed delete operation key=Z5DPD9MNHN2H tag=OA3IIZTN25G6 type=del 2023-01-24T10:52:23.121 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.119+0000 7fbe187898c0 1 > prepared operation key=D92A7W1TH7X0 tag=78W9RTDAU8A6 type=del size=1608434 2023-01-24T10:52:23.122 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.121+0000 7fbe187898c0 1 < completed part key=_multipart_63PDRJCRVEI1.2~D9XZFDI70FFH.0 size=1048576 2023-01-24T10:52:23.125 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.124+0000 7fbe187898c0 1 < completed part key=_multipart_63PDRJCRVEI1.2~D9XZFDI70FFH.1 size=241080 2023-01-24T10:52:23.130 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.129+0000 7fbe187898c0 1 < completed multipart upload key=63PDRJCRVEI1 upload=D9XZFDI70FFH tag=N2HK71FF22LZ type=write size=2338232 2023-01-24T10:52:23.136 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.135+0000 7fbe187898c0 1 > prepared part key=_multipart_D0CGZ9OQK1AM.2~XVKXTHT26MM0.0 size=1048576 2023-01-24T10:52:23.139 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.137+0000 7fbe187898c0 1 > prepared part key=_multipart_D0CGZ9OQK1AM.2~XVKXTHT26MM0.1 size=1048576 2023-01-24T10:52:23.142 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.140+0000 7fbe187898c0 1 > prepared part key=_multipart_D0CGZ9OQK1AM.2~XVKXTHT26MM0.2 size=120706 2023-01-24T10:52:23.142 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.140+0000 7fbe187898c0 1 > prepared multipart upload key=D0CGZ9OQK1AM upload=XVKXTHT26MM0 tag=520U47KO137X type=write size=3266434 2023-01-24T10:52:23.144 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.143+0000 7fbe187898c0 1 < completed delete operation key=D0CGZ9OQK1AM tag=EILE8277S2O1 type=del 2023-01-24T10:52:23.147 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.146+0000 7fbe187898c0 1 > prepared operation key=Z8G7Y4RGVWR8 tag=JAEYU2YBVMMV type=del size=3979125 2023-01-24T10:52:23.149 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.148+0000 7fbe187898c0 1 < completed part key=_multipart_ZMEBN0OQ1IPJ.2~U5DE9INXCWXJ.0 size=1048576 2023-01-24T10:52:23.152 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.151+0000 7fbe187898c0 1 < completed part key=_multipart_ZMEBN0OQ1IPJ.2~U5DE9INXCWXJ.1 size=273500 2023-01-24T10:52:23.156 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.155+0000 7fbe187898c0 1 < completed multipart upload key=ZMEBN0OQ1IPJ upload=U5DE9INXCWXJ tag=79A9ONP946CG type=write size=2370652 2023-01-24T10:52:23.160 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.158+0000 7fbe187898c0 1 > prepared operation key=SNBN0CZ6S9TA tag=QGKH3K5TRRRM type=del size=1880620 2023-01-24T10:52:23.163 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.162+0000 7fbe187898c0 1 < suggested update operation key=YTEKYBINDI7I tag=O5D9N83DHYOG type=del size=1252018 2023-01-24T10:52:23.168 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.167+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~UI07C0A6QRRV.0 size=1048576 2023-01-24T10:52:23.169 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.168+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~UI07C0A6QRRV.1 size=304170 2023-01-24T10:52:23.169 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.168+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=UI07C0A6QRRV tag=LOUET3BOVEG4 type=write size=2401322 2023-01-24T10:52:23.170 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.169+0000 7fbe187898c0 1 < completed delete operation key=3GJVNT6TXTK6 tag=V7U49Y3B8F52 type=del 2023-01-24T10:52:23.172 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.170+0000 7fbe187898c0 1 > prepared operation key=FDZVWZ0V33EB tag=JP73JO70425O type=del size=2179946 2023-01-24T10:52:23.173 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.172+0000 7fbe187898c0 1 < completed part key=_multipart_63PDRJCRVEI1.2~C6LGNGEUIXUW.0 size=1048576 2023-01-24T10:52:23.174 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.173+0000 7fbe187898c0 1 < completed part key=_multipart_63PDRJCRVEI1.2~C6LGNGEUIXUW.1 size=90247 2023-01-24T10:52:23.179 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.178+0000 7fbe187898c0 1 < completed multipart upload key=63PDRJCRVEI1 upload=C6LGNGEUIXUW tag=JINY1AXUP43W type=write size=2187399 2023-01-24T10:52:23.180 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.179+0000 7fbe187898c0 1 > prepared operation key=Z8G7Y4RGVWR8 tag=RY5EOT8YZFC5 type=del size=3041723 2023-01-24T10:52:23.183 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.182+0000 7fbe187898c0 1 < completed delete operation key=TIE8E7PAI3NC tag=GIB5NBLCZZE0 type=del 2023-01-24T10:52:23.186 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.184+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=IJ6MTLCQKIV5 type=write size=183451 2023-01-24T10:52:23.191 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.190+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~GBK80ON6BP1G.0 size=1048576 2023-01-24T10:52:23.192 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.191+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~GBK80ON6BP1G.1 size=1048576 2023-01-24T10:52:23.195 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.193+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~GBK80ON6BP1G.2 size=960413 2023-01-24T10:52:23.198 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.197+0000 7fbe187898c0 1 < completed multipart upload key=FDZVWZ0V33EB upload=GBK80ON6BP1G tag=V6WCCP1H8I0V type=write size=4106141 2023-01-24T10:52:23.204 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.203+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~CG6YSQZPJO72.0 size=1048576 2023-01-24T10:52:23.227 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.225+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~CG6YSQZPJO72.1 size=366015 2023-01-24T10:52:23.227 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.225+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=CG6YSQZPJO72 tag=2R54B36S1SO7 type=write size=2463167 2023-01-24T10:52:23.228 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.227+0000 7fbe187898c0 1 < completed part key=_multipart_1ZN0Y6AS7ISA.2~RUVJ42PFK77Y.0 size=734331 2023-01-24T10:52:23.230 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.229+0000 7fbe187898c0 1 < completed multipart upload key=1ZN0Y6AS7ISA upload=RUVJ42PFK77Y tag=QTOVRDRHJ937 type=write size=1782907 2023-01-24T10:52:23.232 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.230+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=B9CY895R8PE4 type=del size=22329 2023-01-24T10:52:23.233 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.231+0000 7fbe187898c0 1 < completed part key=_multipart_S7XZ2Q32GCMK.2~X5SNF3MK1DLZ.0 size=1048576 2023-01-24T10:52:23.236 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.232+0000 7fbe187898c0 1 < completed part key=_multipart_S7XZ2Q32GCMK.2~X5SNF3MK1DLZ.1 size=852693 2023-01-24T10:52:23.236 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.234+0000 7fbe187898c0 1 < completed multipart upload key=S7XZ2Q32GCMK upload=X5SNF3MK1DLZ tag=UFJNN0TTVUC0 type=write size=2949845 2023-01-24T10:52:23.237 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.235+0000 7fbe187898c0 1 > prepared operation key=D92A7W1TH7X0 tag=JBY1RQG4NVMO type=del size=333565 2023-01-24T10:52:23.238 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.237+0000 7fbe187898c0 1 < canceled operation key=D0CGZ9OQK1AM tag=XU7X1I6K24O3 type=del size=241441 2023-01-24T10:52:23.239 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.238+0000 7fbe187898c0 1 > prepared operation key=5W2UE0AZTOM0 tag=IXKKBBXCYAPS type=del size=1638086 2023-01-24T10:52:23.242 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.241+0000 7fbe187898c0 1 < suggested update operation key=1ZN0Y6AS7ISA tag=2OS4C2E8L350 type=write size=943111 2023-01-24T10:52:23.243 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.242+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=4PMKFIHN7K8K type=del size=3341553 2023-01-24T10:52:23.244 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.243+0000 7fbe187898c0 1 < completed delete operation key=C9J0MO49LDCS tag=4ERXAXKR7YNI type=del 2023-01-24T10:52:23.245 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.244+0000 7fbe187898c0 1 > prepared operation key=XULE5KU5OITV tag=VSDBH4XVBD43 type=del size=1676742 2023-01-24T10:52:23.247 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.245+0000 7fbe187898c0 1 < completed delete operation key=WDQNB5TEJUC6 tag=I38MSTVKAOHU type=del 2023-01-24T10:52:23.248 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.247+0000 7fbe187898c0 1 > prepared operation key=ZMEBN0OQ1IPJ tag=Z620HB6T1C8U type=del size=768010 2023-01-24T10:52:23.249 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.248+0000 7fbe187898c0 1 < completed write operation key=Z4E99NU2FUTH tag=2RGMZRB5E7WK type=write size=173149 2023-01-24T10:52:23.253 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.250+0000 7fbe187898c0 1 > prepared part key=_multipart_RV7FCL2UARTQ.2~K4WG5GP34MG1.0 size=205380 2023-01-24T10:52:23.253 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.250+0000 7fbe187898c0 1 > prepared multipart upload key=RV7FCL2UARTQ upload=K4WG5GP34MG1 tag=XEF7JT4YMSTI type=write size=1253956 2023-01-24T10:52:23.253 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.251+0000 7fbe187898c0 1 < completed delete operation key=D92A7W1TH7X0 tag=78W9RTDAU8A6 type=del 2023-01-24T10:52:23.254 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.253+0000 7fbe187898c0 1 > prepared part key=_multipart_HJGFMANXEK4E.2~BX1MUX0O06UQ.0 size=1048576 2023-01-24T10:52:23.255 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.254+0000 7fbe187898c0 1 > prepared part key=_multipart_HJGFMANXEK4E.2~BX1MUX0O06UQ.1 size=1048576 2023-01-24T10:52:23.256 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.255+0000 7fbe187898c0 1 > prepared part key=_multipart_HJGFMANXEK4E.2~BX1MUX0O06UQ.2 size=172712 2023-01-24T10:52:23.256 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.255+0000 7fbe187898c0 1 > prepared multipart upload key=HJGFMANXEK4E upload=BX1MUX0O06UQ tag=ID9QVZSELSW2 type=write size=3318440 2023-01-24T10:52:23.257 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.256+0000 7fbe187898c0 1 < completed part key=_multipart_D0CGZ9OQK1AM.2~XVKXTHT26MM0.0 size=1048576 2023-01-24T10:52:23.258 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.257+0000 7fbe187898c0 1 < completed part key=_multipart_D0CGZ9OQK1AM.2~XVKXTHT26MM0.1 size=1048576 2023-01-24T10:52:23.259 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.258+0000 7fbe187898c0 1 < completed part key=_multipart_D0CGZ9OQK1AM.2~XVKXTHT26MM0.2 size=120706 2023-01-24T10:52:23.261 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.260+0000 7fbe187898c0 1 < completed multipart upload key=D0CGZ9OQK1AM upload=XVKXTHT26MM0 tag=520U47KO137X type=write size=3266434 2023-01-24T10:52:23.262 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.261+0000 7fbe187898c0 1 > prepared operation key=YTEKYBINDI7I tag=2QJJVRV73K21 type=del size=4185149 2023-01-24T10:52:23.263 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.262+0000 7fbe187898c0 1 < completed delete operation key=Z8G7Y4RGVWR8 tag=JAEYU2YBVMMV type=del 2023-01-24T10:52:23.266 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.263+0000 7fbe187898c0 1 > prepared operation key=C9J0MO49LDCS tag=DV36E04TBNGC type=del size=206394 2023-01-24T10:52:23.266 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.264+0000 7fbe187898c0 1 < completed delete operation key=SNBN0CZ6S9TA tag=QGKH3K5TRRRM type=del 2023-01-24T10:52:23.267 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.265+0000 7fbe187898c0 1 > prepared operation key=TIE8E7PAI3NC tag=LK2IVNJZZY0L type=del size=366260 2023-01-24T10:52:23.268 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.267+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~UI07C0A6QRRV.0 size=1048576 2023-01-24T10:52:23.271 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.270+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~UI07C0A6QRRV.1 size=304170 2023-01-24T10:52:23.275 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.273+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=UI07C0A6QRRV tag=LOUET3BOVEG4 type=write size=2401322 2023-01-24T10:52:23.276 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.275+0000 7fbe187898c0 1 > prepared operation key=RBS906ROG13P tag=FU4WEK91OR1Z type=del size=2731880 2023-01-24T10:52:23.277 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.276+0000 7fbe187898c0 1 < completed delete operation key=FDZVWZ0V33EB tag=JP73JO70425O type=del 2023-01-24T10:52:23.278 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.277+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=5FHF96WXA805 type=del size=618382 2023-01-24T10:52:23.279 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.278+0000 7fbe187898c0 1 < canceled operation key=Z8G7Y4RGVWR8 tag=RY5EOT8YZFC5 type=del size=3041723 2023-01-24T10:52:23.282 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.280+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~B599RV46XYTQ.0 size=1048576 2023-01-24T10:52:23.282 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.281+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~B599RV46XYTQ.1 size=1048576 2023-01-24T10:52:23.283 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.282+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~B599RV46XYTQ.2 size=275782 2023-01-24T10:52:23.283 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.282+0000 7fbe187898c0 1 > prepared multipart upload key=FDZVWZ0V33EB upload=B599RV46XYTQ tag=TLI65FEPTZHL type=write size=3421510 2023-01-24T10:52:23.284 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.283+0000 7fbe187898c0 1 < completed write operation key=NU6FGGI1IUU7 tag=IJ6MTLCQKIV5 type=write size=183451 2023-01-24T10:52:23.286 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.285+0000 7fbe187898c0 1 > prepared part key=_multipart_GNT28K0GNFMJ.2~QCOIITSBC3NR.0 size=1010861 2023-01-24T10:52:23.287 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.285+0000 7fbe187898c0 1 > prepared multipart upload key=GNT28K0GNFMJ upload=QCOIITSBC3NR tag=WU2IYHEFIGVJ type=write size=2059437 2023-01-24T10:52:23.287 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.286+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~CG6YSQZPJO72.0 size=1048576 2023-01-24T10:52:23.288 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.287+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~CG6YSQZPJO72.1 size=366015 2023-01-24T10:52:23.290 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.289+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=CG6YSQZPJO72 tag=2R54B36S1SO7 type=write size=2463167 2023-01-24T10:52:23.292 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.291+0000 7fbe187898c0 1 > prepared part key=_multipart_ZQCEEZ3RIC8F.2~CIXNWFGC6J6R.0 size=1048576 2023-01-24T10:52:23.293 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.292+0000 7fbe187898c0 1 > prepared part key=_multipart_ZQCEEZ3RIC8F.2~CIXNWFGC6J6R.1 size=942806 2023-01-24T10:52:23.293 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.292+0000 7fbe187898c0 1 > prepared multipart upload key=ZQCEEZ3RIC8F upload=CIXNWFGC6J6R tag=E03COP9DZ13I type=write size=3039958 2023-01-24T10:52:23.295 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.293+0000 7fbe187898c0 1 < completed delete operation key=THP93R0E799L tag=B9CY895R8PE4 type=del 2023-01-24T10:52:23.296 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.294+0000 7fbe187898c0 1 > prepared operation key=ZQCEEZ3RIC8F tag=44PRO1CPBW96 type=del size=262104 2023-01-24T10:52:23.298 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.297+0000 7fbe187898c0 1 < suggested remove operation key=D92A7W1TH7X0 tag=JBY1RQG4NVMO type=del 2023-01-24T10:52:23.300 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.299+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~UKXU0VXBGH3R.0 size=682992 2023-01-24T10:52:23.301 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.299+0000 7fbe187898c0 1 > prepared multipart upload key=FDZVWZ0V33EB upload=UKXU0VXBGH3R tag=VOJH289LEI70 type=write size=1731568 2023-01-24T10:52:23.301 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.300+0000 7fbe187898c0 1 < canceled operation key=5W2UE0AZTOM0 tag=IXKKBBXCYAPS type=del size=1638086 2023-01-24T10:52:23.302 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.301+0000 7fbe187898c0 1 > prepared operation key=63PDRJCRVEI1 tag=HJ3S5BB1NNJP type=del size=3572188 2023-01-24T10:52:23.305 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.303+0000 7fbe187898c0 1 < suggested update operation key=THP93R0E799L tag=4PMKFIHN7K8K type=del size=3341553 2023-01-24T10:52:23.306 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.304+0000 7fbe187898c0 1 > prepared operation key=YTEKYBINDI7I tag=ISVLUBREPY15 type=del size=1720114 2023-01-24T10:52:23.308 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.307+0000 7fbe187898c0 1 < suggested remove operation key=XULE5KU5OITV tag=VSDBH4XVBD43 type=del 2023-01-24T10:52:23.309 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.308+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=J3UDCHANX26Y type=del size=553329 2023-01-24T10:52:23.312 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.311+0000 7fbe187898c0 1 < completed delete operation key=ZMEBN0OQ1IPJ tag=Z620HB6T1C8U type=del 2023-01-24T10:52:23.315 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.314+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=OMLBGXWCHX6G type=del size=1545905 2023-01-24T10:52:23.318 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.316+0000 7fbe187898c0 1 < completed part key=_multipart_RV7FCL2UARTQ.2~K4WG5GP34MG1.0 size=205380 2023-01-24T10:52:23.323 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.322+0000 7fbe187898c0 1 < completed multipart upload key=RV7FCL2UARTQ upload=K4WG5GP34MG1 tag=XEF7JT4YMSTI type=write size=1253956 2023-01-24T10:52:23.326 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.325+0000 7fbe187898c0 1 > prepared operation key=0VR1WPQLHOV0 tag=VPDAODID4QV3 type=del size=1424437 2023-01-24T10:52:23.329 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.328+0000 7fbe187898c0 1 < completed part key=_multipart_HJGFMANXEK4E.2~BX1MUX0O06UQ.0 size=1048576 2023-01-24T10:52:23.333 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.331+0000 7fbe187898c0 1 < completed part key=_multipart_HJGFMANXEK4E.2~BX1MUX0O06UQ.1 size=1048576 2023-01-24T10:52:23.335 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.334+0000 7fbe187898c0 1 < completed part key=_multipart_HJGFMANXEK4E.2~BX1MUX0O06UQ.2 size=172712 2023-01-24T10:52:23.340 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.339+0000 7fbe187898c0 1 < completed multipart upload key=HJGFMANXEK4E upload=BX1MUX0O06UQ tag=ID9QVZSELSW2 type=write size=3318440 2023-01-24T10:52:23.346 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.345+0000 7fbe187898c0 1 > prepared part key=_multipart_63PDRJCRVEI1.2~DAL6ZVTKBTB5.0 size=1048576 2023-01-24T10:52:23.349 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.347+0000 7fbe187898c0 1 > prepared part key=_multipart_63PDRJCRVEI1.2~DAL6ZVTKBTB5.1 size=830794 2023-01-24T10:52:23.349 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.347+0000 7fbe187898c0 1 > prepared multipart upload key=63PDRJCRVEI1 upload=DAL6ZVTKBTB5 tag=D2PESTPSXYKU type=write size=2927946 2023-01-24T10:52:23.351 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.350+0000 7fbe187898c0 1 < completed delete operation key=YTEKYBINDI7I tag=2QJJVRV73K21 type=del 2023-01-24T10:52:23.354 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.353+0000 7fbe187898c0 1 > prepared part key=_multipart_A0DJSLNVH25F.2~VCGN575T45FS.0 size=1048576 2023-01-24T10:52:23.355 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.354+0000 7fbe187898c0 1 > prepared part key=_multipart_A0DJSLNVH25F.2~VCGN575T45FS.1 size=1048576 2023-01-24T10:52:23.356 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.354+0000 7fbe187898c0 1 > prepared part key=_multipart_A0DJSLNVH25F.2~VCGN575T45FS.2 size=288006 2023-01-24T10:52:23.356 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.354+0000 7fbe187898c0 1 > prepared multipart upload key=A0DJSLNVH25F upload=VCGN575T45FS tag=YOQWUP43YOEJ type=write size=3433734 2023-01-24T10:52:23.357 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.356+0000 7fbe187898c0 1 < completed delete operation key=C9J0MO49LDCS tag=DV36E04TBNGC type=del 2023-01-24T10:52:23.363 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.362+0000 7fbe187898c0 1 > prepared operation key=Z5DPD9MNHN2H tag=1P2AXIN96IK0 type=del size=3717738 2023-01-24T10:52:23.366 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.364+0000 7fbe187898c0 1 < suggested remove operation key=TIE8E7PAI3NC tag=LK2IVNJZZY0L type=del 2023-01-24T10:52:23.368 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.367+0000 7fbe187898c0 1 > prepared part key=_multipart_WDQNB5TEJUC6.2~7XGALSRVCMXW.0 size=1048576 2023-01-24T10:52:23.369 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.367+0000 7fbe187898c0 1 > prepared part key=_multipart_WDQNB5TEJUC6.2~7XGALSRVCMXW.1 size=769255 2023-01-24T10:52:23.369 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.367+0000 7fbe187898c0 1 > prepared multipart upload key=WDQNB5TEJUC6 upload=7XGALSRVCMXW tag=QZGE4W85LYIZ type=write size=2866407 2023-01-24T10:52:23.370 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.369+0000 7fbe187898c0 1 < completed delete operation key=RBS906ROG13P tag=FU4WEK91OR1Z type=del 2023-01-24T10:52:23.372 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.371+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~ZBMJQH3ZV5TG.0 size=1048576 2023-01-24T10:52:23.373 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.372+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~ZBMJQH3ZV5TG.1 size=657552 2023-01-24T10:52:23.373 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.372+0000 7fbe187898c0 1 > prepared multipart upload key=Z5DPD9MNHN2H upload=ZBMJQH3ZV5TG tag=7T3PIRKA6S24 type=write size=2754704 2023-01-24T10:52:23.374 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.373+0000 7fbe187898c0 1 < completed delete operation key=YGC2UEGMODS8 tag=5FHF96WXA805 type=del 2023-01-24T10:52:23.377 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.376+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=288EFOGSQZ8O type=write size=776297 2023-01-24T10:52:23.379 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.377+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~B599RV46XYTQ.0 size=1048576 2023-01-24T10:52:23.379 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.377+0000 7fbe187898c0 1 < canceled part key=_multipart_FDZVWZ0V33EB.2~B599RV46XYTQ.1 size=1048576 2023-01-24T10:52:23.379 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.378+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~B599RV46XYTQ.2 size=275782 2023-01-24T10:52:23.381 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.380+0000 7fbe187898c0 1 < canceled multipart upload key=FDZVWZ0V33EB upload=B599RV46XYTQ tag=TLI65FEPTZHL type=write size=3421510 2023-01-24T10:52:23.383 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.381+0000 7fbe187898c0 1 > prepared operation key=K1AKVEEHDJEG tag=22DDSF1TGXGH type=write size=155210 2023-01-24T10:52:23.384 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.382+0000 7fbe187898c0 1 < completed part key=_multipart_GNT28K0GNFMJ.2~QCOIITSBC3NR.0 size=1010861 2023-01-24T10:52:23.385 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.384+0000 7fbe187898c0 1 < completed multipart upload key=GNT28K0GNFMJ upload=QCOIITSBC3NR tag=WU2IYHEFIGVJ type=write size=2059437 2023-01-24T10:52:23.388 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.386+0000 7fbe187898c0 1 > prepared part key=_multipart_1ZN0Y6AS7ISA.2~UTR4GD94MEQS.0 size=1048576 2023-01-24T10:52:23.391 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.390+0000 7fbe187898c0 1 > prepared part key=_multipart_1ZN0Y6AS7ISA.2~UTR4GD94MEQS.1 size=1048576 2023-01-24T10:52:23.394 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.392+0000 7fbe187898c0 1 > prepared part key=_multipart_1ZN0Y6AS7ISA.2~UTR4GD94MEQS.2 size=265023 2023-01-24T10:52:23.394 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.392+0000 7fbe187898c0 1 > prepared multipart upload key=1ZN0Y6AS7ISA upload=UTR4GD94MEQS tag=T21B66KHOO9J type=write size=3410751 2023-01-24T10:52:23.396 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.395+0000 7fbe187898c0 1 < completed part key=_multipart_ZQCEEZ3RIC8F.2~CIXNWFGC6J6R.0 size=1048576 2023-01-24T10:52:23.399 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.397+0000 7fbe187898c0 1 < completed part key=_multipart_ZQCEEZ3RIC8F.2~CIXNWFGC6J6R.1 size=942806 2023-01-24T10:52:23.401 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.399+0000 7fbe187898c0 1 < completed multipart upload key=ZQCEEZ3RIC8F upload=CIXNWFGC6J6R tag=E03COP9DZ13I type=write size=3039958 2023-01-24T10:52:23.402 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.401+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=ME5KB6CW7QTV type=del size=3433910 2023-01-24T10:52:23.404 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.403+0000 7fbe187898c0 1 < suggested update operation key=ZQCEEZ3RIC8F tag=44PRO1CPBW96 type=del size=262104 2023-01-24T10:52:23.406 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.404+0000 7fbe187898c0 1 > prepared operation key=1ZN0Y6AS7ISA tag=ZLS2MTTCTKYH type=write size=523952 2023-01-24T10:52:23.407 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.405+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~UKXU0VXBGH3R.0 size=682992 2023-01-24T10:52:23.409 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.408+0000 7fbe187898c0 1 < completed multipart upload key=FDZVWZ0V33EB upload=UKXU0VXBGH3R tag=VOJH289LEI70 type=write size=1731568 2023-01-24T10:52:23.410 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.409+0000 7fbe187898c0 1 > prepared operation key=S7XZ2Q32GCMK tag=EC2FZH94LH22 type=write size=519019 2023-01-24T10:52:23.411 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.410+0000 7fbe187898c0 1 < completed delete operation key=63PDRJCRVEI1 tag=HJ3S5BB1NNJP type=del 2023-01-24T10:52:23.413 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.412+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~LSNELXYWP1WK.0 size=1048576 2023-01-24T10:52:23.414 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.413+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~LSNELXYWP1WK.1 size=772073 2023-01-24T10:52:23.415 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.413+0000 7fbe187898c0 1 > prepared multipart upload key=9UV7IWZOV627 upload=LSNELXYWP1WK tag=1OD87NJ0KNJL type=write size=2869225 2023-01-24T10:52:23.415 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.414+0000 7fbe187898c0 1 < completed delete operation key=YTEKYBINDI7I tag=ISVLUBREPY15 type=del 2023-01-24T10:52:23.417 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.416+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=ICLG2N2HW117 type=del size=1435822 2023-01-24T10:52:23.419 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.418+0000 7fbe187898c0 1 < suggested remove operation key=MQAFAWM6WGH1 tag=J3UDCHANX26Y type=del 2023-01-24T10:52:23.421 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.419+0000 7fbe187898c0 1 > prepared operation key=GNT28K0GNFMJ tag=1HCNY76TM30R type=del size=969250 2023-01-24T10:52:23.423 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.421+0000 7fbe187898c0 1 < suggested remove operation key=VBIA0USNT4OA tag=OMLBGXWCHX6G type=del 2023-01-24T10:52:23.425 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.424+0000 7fbe187898c0 1 > prepared part key=_multipart_1ZN0Y6AS7ISA.2~R4S6BZVCLP25.0 size=1048576 2023-01-24T10:52:23.426 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.425+0000 7fbe187898c0 1 > prepared part key=_multipart_1ZN0Y6AS7ISA.2~R4S6BZVCLP25.1 size=1048576 2023-01-24T10:52:23.429 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.427+0000 7fbe187898c0 1 > prepared part key=_multipart_1ZN0Y6AS7ISA.2~R4S6BZVCLP25.2 size=904079 2023-01-24T10:52:23.429 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.427+0000 7fbe187898c0 1 > prepared multipart upload key=1ZN0Y6AS7ISA upload=R4S6BZVCLP25 tag=YMMZ10I51ZH2 type=write size=4049807 2023-01-24T10:52:23.434 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.433+0000 7fbe187898c0 1 < suggested remove operation key=0VR1WPQLHOV0 tag=VPDAODID4QV3 type=del 2023-01-24T10:52:23.437 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.435+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=GKOMF86ZVY58 type=del size=2126700 2023-01-24T10:52:23.439 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.438+0000 7fbe187898c0 1 < completed part key=_multipart_63PDRJCRVEI1.2~DAL6ZVTKBTB5.0 size=1048576 2023-01-24T10:52:23.442 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.441+0000 7fbe187898c0 1 < completed part key=_multipart_63PDRJCRVEI1.2~DAL6ZVTKBTB5.1 size=830794 2023-01-24T10:52:23.447 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.446+0000 7fbe187898c0 1 < completed multipart upload key=63PDRJCRVEI1 upload=DAL6ZVTKBTB5 tag=D2PESTPSXYKU type=write size=2927946 2023-01-24T10:52:23.449 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.447+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=FP51SK88T3JM type=del size=923221 2023-01-24T10:52:23.451 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.450+0000 7fbe187898c0 1 < completed part key=_multipart_A0DJSLNVH25F.2~VCGN575T45FS.0 size=1048576 2023-01-24T10:52:23.454 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.453+0000 7fbe187898c0 1 < completed part key=_multipart_A0DJSLNVH25F.2~VCGN575T45FS.1 size=1048576 2023-01-24T10:52:23.457 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.456+0000 7fbe187898c0 1 < completed part key=_multipart_A0DJSLNVH25F.2~VCGN575T45FS.2 size=288006 2023-01-24T10:52:23.459 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.458+0000 7fbe187898c0 1 < completed multipart upload key=A0DJSLNVH25F upload=VCGN575T45FS tag=YOQWUP43YOEJ type=write size=3433734 2023-01-24T10:52:23.461 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.460+0000 7fbe187898c0 1 > prepared part key=_multipart_C9J0MO49LDCS.2~D23DGBKHFPCT.0 size=900499 2023-01-24T10:52:23.461 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.460+0000 7fbe187898c0 1 > prepared multipart upload key=C9J0MO49LDCS upload=D23DGBKHFPCT tag=Q1K5Q3DYAMDV type=write size=1949075 2023-01-24T10:52:23.462 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.461+0000 7fbe187898c0 1 < completed delete operation key=Z5DPD9MNHN2H tag=1P2AXIN96IK0 type=del 2023-01-24T10:52:23.463 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.462+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=2OFS8BEPQFSQ type=del size=821832 2023-01-24T10:52:23.465 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.463+0000 7fbe187898c0 1 < completed part key=_multipart_WDQNB5TEJUC6.2~7XGALSRVCMXW.0 size=1048576 2023-01-24T10:52:23.465 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.463+0000 7fbe187898c0 1 < canceled part key=_multipart_WDQNB5TEJUC6.2~7XGALSRVCMXW.1 size=769255 2023-01-24T10:52:23.467 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.466+0000 7fbe187898c0 1 < canceled multipart upload key=WDQNB5TEJUC6 upload=7XGALSRVCMXW tag=QZGE4W85LYIZ type=write size=2866407 2023-01-24T10:52:23.468 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.467+0000 7fbe187898c0 1 > prepared operation key=A0DJSLNVH25F tag=VHDY6548ZJS4 type=del size=691540 2023-01-24T10:52:23.469 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.468+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~ZBMJQH3ZV5TG.0 size=1048576 2023-01-24T10:52:23.470 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.469+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~ZBMJQH3ZV5TG.1 size=657552 2023-01-24T10:52:23.473 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.471+0000 7fbe187898c0 1 < completed multipart upload key=Z5DPD9MNHN2H upload=ZBMJQH3ZV5TG tag=7T3PIRKA6S24 type=write size=2754704 2023-01-24T10:52:23.474 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.472+0000 7fbe187898c0 1 > prepared operation key=GNT28K0GNFMJ tag=9IL69Q6XMVYD type=del size=1388609 2023-01-24T10:52:23.477 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.475+0000 7fbe187898c0 1 < completed write operation key=NU6FGGI1IUU7 tag=288EFOGSQZ8O type=write size=776297 2023-01-24T10:52:23.478 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.477+0000 7fbe187898c0 1 > prepared operation key=D92A7W1TH7X0 tag=4EYIOTX68QA0 type=del size=2109353 2023-01-24T10:52:23.479 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.478+0000 7fbe187898c0 1 < completed write operation key=K1AKVEEHDJEG tag=22DDSF1TGXGH type=write size=155210 2023-01-24T10:52:23.480 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.479+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=5DMHZHTAQVBD type=del size=2272919 2023-01-24T10:52:23.481 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.479+0000 7fbe187898c0 1 < canceled part key=_multipart_1ZN0Y6AS7ISA.2~UTR4GD94MEQS.0 size=1048576 2023-01-24T10:52:23.482 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.480+0000 7fbe187898c0 1 < completed part key=_multipart_1ZN0Y6AS7ISA.2~UTR4GD94MEQS.1 size=1048576 2023-01-24T10:52:23.483 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.481+0000 7fbe187898c0 1 < completed part key=_multipart_1ZN0Y6AS7ISA.2~UTR4GD94MEQS.2 size=265023 2023-01-24T10:52:23.485 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.484+0000 7fbe187898c0 1 < canceled multipart upload key=1ZN0Y6AS7ISA upload=UTR4GD94MEQS tag=T21B66KHOO9J type=write size=3410751 2023-01-24T10:52:23.486 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.485+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=YQM3HQNTBHVD type=write size=754285 2023-01-24T10:52:23.487 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.486+0000 7fbe187898c0 -1 < failed to complete operation key=VBIA0USNT4OA tag=ME5KB6CW7QTV type=del size=3433910: (22) Invalid argument 2023-01-24T10:52:23.490 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.489+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~PCEA302YUV6C.0 size=168810 2023-01-24T10:52:23.490 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.489+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=PCEA302YUV6C tag=GHE0L4HOTQ65 type=write size=1217386 2023-01-24T10:52:23.491 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.490+0000 7fbe187898c0 1 < completed write operation key=1ZN0Y6AS7ISA tag=ZLS2MTTCTKYH type=write size=523952 2023-01-24T10:52:23.492 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.491+0000 7fbe187898c0 1 > prepared operation key=YTEKYBINDI7I tag=9S4OVN1139TV type=write size=496092 2023-01-24T10:52:23.494 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.492+0000 7fbe187898c0 1 < canceled operation key=S7XZ2Q32GCMK tag=EC2FZH94LH22 type=write size=519019 2023-01-24T10:52:23.496 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.495+0000 7fbe187898c0 1 > prepared operation key=GNT28K0GNFMJ tag=MXM6T0VVST7F type=del size=2628317 2023-01-24T10:52:23.499 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.498+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~LSNELXYWP1WK.0 size=1048576 2023-01-24T10:52:23.502 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.501+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~LSNELXYWP1WK.1 size=772073 2023-01-24T10:52:23.504 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.503+0000 7fbe187898c0 1 < completed multipart upload key=9UV7IWZOV627 upload=LSNELXYWP1WK tag=1OD87NJ0KNJL type=write size=2869225 2023-01-24T10:52:23.506 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.505+0000 7fbe187898c0 1 > prepared part key=_multipart_D92A7W1TH7X0.2~Q9WQ3ED3T7NC.0 size=1048576 2023-01-24T10:52:23.507 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.506+0000 7fbe187898c0 1 > prepared part key=_multipart_D92A7W1TH7X0.2~Q9WQ3ED3T7NC.1 size=1014266 2023-01-24T10:52:23.507 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.506+0000 7fbe187898c0 1 > prepared multipart upload key=D92A7W1TH7X0 upload=Q9WQ3ED3T7NC tag=QRA5YPITEZ1P type=write size=3111418 2023-01-24T10:52:23.508 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.507+0000 7fbe187898c0 1 < completed delete operation key=NU6FGGI1IUU7 tag=ICLG2N2HW117 type=del 2023-01-24T10:52:23.510 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.509+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~P2T5XHX5X1NM.0 size=1048576 2023-01-24T10:52:23.512 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.510+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~P2T5XHX5X1NM.1 size=1033734 2023-01-24T10:52:23.513 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.510+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=P2T5XHX5X1NM tag=7D9W62VN30KB type=write size=3130886 2023-01-24T10:52:23.514 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.512+0000 7fbe187898c0 1 < suggested remove operation key=GNT28K0GNFMJ tag=1HCNY76TM30R type=del 2023-01-24T10:52:23.516 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.515+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~0F0RZLIQVAZO.0 size=1048576 2023-01-24T10:52:23.517 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.516+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~0F0RZLIQVAZO.1 size=1048576 2023-01-24T10:52:23.518 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.517+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~0F0RZLIQVAZO.2 size=244852 2023-01-24T10:52:23.518 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.517+0000 7fbe187898c0 1 > prepared multipart upload key=Z5DPD9MNHN2H upload=0F0RZLIQVAZO tag=TWUHI1D0F26F type=write size=3390580 2023-01-24T10:52:23.519 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.518+0000 7fbe187898c0 1 < completed part key=_multipart_1ZN0Y6AS7ISA.2~R4S6BZVCLP25.0 size=1048576 2023-01-24T10:52:23.519 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.518+0000 7fbe187898c0 1 < canceled part key=_multipart_1ZN0Y6AS7ISA.2~R4S6BZVCLP25.1 size=1048576 2023-01-24T10:52:23.520 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.519+0000 7fbe187898c0 1 < completed part key=_multipart_1ZN0Y6AS7ISA.2~R4S6BZVCLP25.2 size=904079 2023-01-24T10:52:23.522 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.521+0000 7fbe187898c0 1 < canceled multipart upload key=1ZN0Y6AS7ISA upload=R4S6BZVCLP25 tag=YMMZ10I51ZH2 type=write size=4049807 2023-01-24T10:52:23.525 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.523+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~UN2OYKIUALTU.0 size=1048576 2023-01-24T10:52:23.525 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.524+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~UN2OYKIUALTU.1 size=1048576 2023-01-24T10:52:23.526 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.525+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~UN2OYKIUALTU.2 size=29281 2023-01-24T10:52:23.527 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.525+0000 7fbe187898c0 1 > prepared multipart upload key=9UV7IWZOV627 upload=UN2OYKIUALTU tag=LI7TUFFC4NBW type=write size=3175009 2023-01-24T10:52:23.528 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.527+0000 7fbe187898c0 1 < completed delete operation key=YGC2UEGMODS8 tag=GKOMF86ZVY58 type=del 2023-01-24T10:52:23.530 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.529+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~K61B222981CO.0 size=1048576 2023-01-24T10:52:23.533 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.532+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~K61B222981CO.1 size=22087 2023-01-24T10:52:23.533 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.532+0000 7fbe187898c0 1 > prepared multipart upload key=TIE8E7PAI3NC upload=K61B222981CO tag=WI631TPKGTUT type=write size=2119239 2023-01-24T10:52:23.535 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.533+0000 7fbe187898c0 1 < completed delete operation key=YGC2UEGMODS8 tag=FP51SK88T3JM type=del 2023-01-24T10:52:23.537 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.536+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~9S5KVPSMODMF.0 size=50905 2023-01-24T10:52:23.537 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.536+0000 7fbe187898c0 1 > prepared multipart upload key=3GJVNT6TXTK6 upload=9S5KVPSMODMF tag=LOBGFC523VTO type=write size=1099481 2023-01-24T10:52:23.538 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.537+0000 7fbe187898c0 1 < completed part key=_multipart_C9J0MO49LDCS.2~D23DGBKHFPCT.0 size=900499 2023-01-24T10:52:23.541 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.539+0000 7fbe187898c0 1 < completed multipart upload key=C9J0MO49LDCS upload=D23DGBKHFPCT tag=Q1K5Q3DYAMDV type=write size=1949075 2023-01-24T10:52:23.542 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.541+0000 7fbe187898c0 1 > prepared operation key=3GJVNT6TXTK6 tag=J143FADIMOE8 type=del size=1431922 2023-01-24T10:52:23.546 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.542+0000 7fbe187898c0 1 < completed delete operation key=MQAFAWM6WGH1 tag=2OFS8BEPQFSQ type=del 2023-01-24T10:52:23.546 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.543+0000 7fbe187898c0 1 > prepared operation key=1ZN0Y6AS7ISA tag=VTL43TJHQQKQ type=del size=88448 2023-01-24T10:52:23.546 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.545+0000 7fbe187898c0 1 < completed delete operation key=A0DJSLNVH25F tag=VHDY6548ZJS4 type=del 2023-01-24T10:52:23.548 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.547+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~35E6S04LOWUE.0 size=801850 2023-01-24T10:52:23.548 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.547+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=35E6S04LOWUE tag=WS02TCCWYCRL type=write size=1850426 2023-01-24T10:52:23.551 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.550+0000 7fbe187898c0 -1 < failed to complete operation key=GNT28K0GNFMJ tag=9IL69Q6XMVYD type=del size=1388609: (22) Invalid argument 2023-01-24T10:52:23.554 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.553+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=Z6I0G4YD53EG type=del size=3476075 2023-01-24T10:52:23.556 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.555+0000 7fbe187898c0 1 < completed delete operation key=D92A7W1TH7X0 tag=4EYIOTX68QA0 type=del 2023-01-24T10:52:23.559 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.558+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=PTU0XVHMWV2S type=del size=1434833 2023-01-24T10:52:23.563 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.561+0000 7fbe187898c0 1 < suggested remove operation key=MQAFAWM6WGH1 tag=5DMHZHTAQVBD type=del 2023-01-24T10:52:23.568 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.567+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~V1ZWT0XQ7LET.0 size=1048576 2023-01-24T10:52:23.571 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.569+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~V1ZWT0XQ7LET.1 size=1048576 2023-01-24T10:52:23.574 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.572+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~V1ZWT0XQ7LET.2 size=495855 2023-01-24T10:52:23.574 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.572+0000 7fbe187898c0 1 > prepared multipart upload key=5W2UE0AZTOM0 upload=V1ZWT0XQ7LET tag=69QJ3IP4MYCW type=write size=3641583 2023-01-24T10:52:23.575 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.574+0000 7fbe187898c0 1 < completed write operation key=YGC2UEGMODS8 tag=YQM3HQNTBHVD type=write size=754285 2023-01-24T10:52:23.576 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.575+0000 7fbe187898c0 1 > prepared operation key=3GJVNT6TXTK6 tag=OKN1I6DMS50G type=del size=1551132 2023-01-24T10:52:23.577 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.576+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~PCEA302YUV6C.0 size=168810 2023-01-24T10:52:23.580 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.578+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=PCEA302YUV6C tag=GHE0L4HOTQ65 type=write size=1217386 2023-01-24T10:52:23.581 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.580+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=IWKT020Z48W4 type=write size=473592 2023-01-24T10:52:23.584 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.582+0000 7fbe187898c0 1 < canceled operation key=YTEKYBINDI7I tag=9S4OVN1139TV type=write size=496092 2023-01-24T10:52:23.586 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.585+0000 7fbe187898c0 1 > prepared operation key=ZMEBN0OQ1IPJ tag=3OQ1LCHQQPFX type=write size=1031709 2023-01-24T10:52:23.589 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.588+0000 7fbe187898c0 -1 < failed to complete operation key=GNT28K0GNFMJ tag=MXM6T0VVST7F type=del size=2628317: (22) Invalid argument 2023-01-24T10:52:23.592 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.590+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=J98PH3LW25W8 type=write size=77793 2023-01-24T10:52:23.595 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.594+0000 7fbe187898c0 1 < completed part key=_multipart_D92A7W1TH7X0.2~Q9WQ3ED3T7NC.0 size=1048576 2023-01-24T10:52:23.597 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.596+0000 7fbe187898c0 1 < completed part key=_multipart_D92A7W1TH7X0.2~Q9WQ3ED3T7NC.1 size=1014266 2023-01-24T10:52:23.603 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.601+0000 7fbe187898c0 1 < completed multipart upload key=D92A7W1TH7X0 upload=Q9WQ3ED3T7NC tag=QRA5YPITEZ1P type=write size=3111418 2023-01-24T10:52:23.605 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.603+0000 7fbe187898c0 1 > prepared operation key=SNBN0CZ6S9TA tag=TGAPUZ6Z234K type=write size=467403 2023-01-24T10:52:23.605 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.604+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~P2T5XHX5X1NM.0 size=1048576 2023-01-24T10:52:23.606 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.605+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~P2T5XHX5X1NM.1 size=1033734 2023-01-24T10:52:23.611 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.607+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=P2T5XHX5X1NM tag=7D9W62VN30KB type=write size=3130886 2023-01-24T10:52:23.612 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.610+0000 7fbe187898c0 1 > prepared part key=_multipart_K1AKVEEHDJEG.2~LAA3608930ZA.0 size=63218 2023-01-24T10:52:23.612 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.610+0000 7fbe187898c0 1 > prepared multipart upload key=K1AKVEEHDJEG upload=LAA3608930ZA tag=YOJ6FY0QQSO7 type=write size=1111794 2023-01-24T10:52:23.613 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.611+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~0F0RZLIQVAZO.0 size=1048576 2023-01-24T10:52:23.613 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.612+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~0F0RZLIQVAZO.1 size=1048576 2023-01-24T10:52:23.614 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.613+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~0F0RZLIQVAZO.2 size=244852 2023-01-24T10:52:23.616 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.615+0000 7fbe187898c0 1 < completed multipart upload key=Z5DPD9MNHN2H upload=0F0RZLIQVAZO tag=TWUHI1D0F26F type=write size=3390580 2023-01-24T10:52:23.619 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.617+0000 7fbe187898c0 1 > prepared part key=_multipart_ZQCEEZ3RIC8F.2~D0CR6LC5R7ZS.0 size=535759 2023-01-24T10:52:23.619 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.617+0000 7fbe187898c0 1 > prepared multipart upload key=ZQCEEZ3RIC8F upload=D0CR6LC5R7ZS tag=KANW300Q2D20 type=write size=1584335 2023-01-24T10:52:23.620 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.619+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~UN2OYKIUALTU.0 size=1048576 2023-01-24T10:52:23.621 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.620+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~UN2OYKIUALTU.1 size=1048576 2023-01-24T10:52:23.622 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.621+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~UN2OYKIUALTU.2 size=29281 2023-01-24T10:52:23.624 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.623+0000 7fbe187898c0 1 < completed multipart upload key=9UV7IWZOV627 upload=UN2OYKIUALTU tag=LI7TUFFC4NBW type=write size=3175009 2023-01-24T10:52:23.625 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.624+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=ETOKTDOQLFZ0 type=del size=3288186 2023-01-24T10:52:23.627 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.625+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~K61B222981CO.0 size=1048576 2023-01-24T10:52:23.628 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.626+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~K61B222981CO.1 size=22087 2023-01-24T10:52:23.630 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.628+0000 7fbe187898c0 1 < completed multipart upload key=TIE8E7PAI3NC upload=K61B222981CO tag=WI631TPKGTUT type=write size=2119239 2023-01-24T10:52:23.631 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.630+0000 7fbe187898c0 1 > prepared operation key=D0CGZ9OQK1AM tag=M1S1VMU0XEB1 type=del size=1136983 2023-01-24T10:52:23.632 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.631+0000 7fbe187898c0 1 < completed part key=_multipart_3GJVNT6TXTK6.2~9S5KVPSMODMF.0 size=50905 2023-01-24T10:52:23.634 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.633+0000 7fbe187898c0 1 < completed multipart upload key=3GJVNT6TXTK6 upload=9S5KVPSMODMF tag=LOBGFC523VTO type=write size=1099481 2023-01-24T10:52:23.636 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.635+0000 7fbe187898c0 1 > prepared operation key=D92A7W1TH7X0 tag=H4ZK90Y4E363 type=write size=698403 2023-01-24T10:52:23.639 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.637+0000 7fbe187898c0 1 < suggested update operation key=3GJVNT6TXTK6 tag=J143FADIMOE8 type=del size=1431922 2023-01-24T10:52:23.640 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.639+0000 7fbe187898c0 1 > prepared operation key=1ZN0Y6AS7ISA tag=YWR368DB8L32 type=del size=2571958 2023-01-24T10:52:23.641 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.640+0000 7fbe187898c0 1 < completed delete operation key=1ZN0Y6AS7ISA tag=VTL43TJHQQKQ type=del 2023-01-24T10:52:23.643 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.642+0000 7fbe187898c0 1 > prepared part key=_multipart_NU6FGGI1IUU7.2~I6QBJY8NXCFP.0 size=641001 2023-01-24T10:52:23.644 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.642+0000 7fbe187898c0 1 > prepared multipart upload key=NU6FGGI1IUU7 upload=I6QBJY8NXCFP tag=C121TJQOT1SD type=write size=1689577 2023-01-24T10:52:23.644 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.642+0000 7fbe187898c0 1 < canceled part key=_multipart_VBIA0USNT4OA.2~35E6S04LOWUE.0 size=801850 2023-01-24T10:52:23.647 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.646+0000 7fbe187898c0 1 < canceled multipart upload key=VBIA0USNT4OA upload=35E6S04LOWUE tag=WS02TCCWYCRL type=write size=1850426 2023-01-24T10:52:23.649 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.648+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=ZL5MENCZTI05 type=del size=716145 2023-01-24T10:52:23.651 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.650+0000 7fbe187898c0 1 < canceled operation key=VBIA0USNT4OA tag=Z6I0G4YD53EG type=del size=3476075 2023-01-24T10:52:23.657 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.655+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~V09EN41KOCZV.0 size=980116 2023-01-24T10:52:23.657 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.655+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=V09EN41KOCZV tag=HFDE598918PB type=write size=2028692 2023-01-24T10:52:23.662 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.661+0000 7fbe187898c0 1 < suggested remove operation key=NU6FGGI1IUU7 tag=PTU0XVHMWV2S type=del 2023-01-24T10:52:23.666 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.665+0000 7fbe187898c0 1 > prepared part key=_multipart_C9J0MO49LDCS.2~CEEI7C21J28I.0 size=1048576 2023-01-24T10:52:23.669 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.667+0000 7fbe187898c0 1 > prepared part key=_multipart_C9J0MO49LDCS.2~CEEI7C21J28I.1 size=1048576 2023-01-24T10:52:23.671 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.670+0000 7fbe187898c0 1 > prepared part key=_multipart_C9J0MO49LDCS.2~CEEI7C21J28I.2 size=538261 2023-01-24T10:52:23.671 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.670+0000 7fbe187898c0 1 > prepared multipart upload key=C9J0MO49LDCS upload=CEEI7C21J28I tag=9U685Y5DBZF4 type=write size=3683989 2023-01-24T10:52:23.674 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.673+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~V1ZWT0XQ7LET.0 size=1048576 2023-01-24T10:52:23.675 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.674+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~V1ZWT0XQ7LET.1 size=1048576 2023-01-24T10:52:23.676 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.675+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~V1ZWT0XQ7LET.2 size=495855 2023-01-24T10:52:23.678 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.677+0000 7fbe187898c0 1 < completed multipart upload key=5W2UE0AZTOM0 upload=V1ZWT0XQ7LET tag=69QJ3IP4MYCW type=write size=3641583 2023-01-24T10:52:23.680 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.678+0000 7fbe187898c0 1 > prepared operation key=A0DJSLNVH25F tag=QNQF4B09GAU6 type=del size=3699858 2023-01-24T10:52:23.681 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.679+0000 7fbe187898c0 -1 < failed to complete operation key=3GJVNT6TXTK6 tag=OKN1I6DMS50G type=del size=1551132: (22) Invalid argument 2023-01-24T10:52:23.682 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.681+0000 7fbe187898c0 1 > prepared operation key=C9J0MO49LDCS tag=L250DWRW2VFW type=del size=562726 2023-01-24T10:52:23.683 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.682+0000 7fbe187898c0 -1 < failed to complete operation key=NU6FGGI1IUU7 tag=IWKT020Z48W4 type=write size=473592: (22) Invalid argument 2023-01-24T10:52:23.685 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.683+0000 7fbe187898c0 1 > prepared operation key=ZMEBN0OQ1IPJ tag=DG5QK7LQDFP2 type=del size=2656609 2023-01-24T10:52:23.686 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.684+0000 7fbe187898c0 1 < completed write operation key=ZMEBN0OQ1IPJ tag=3OQ1LCHQQPFX type=write size=1031709 2023-01-24T10:52:23.687 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.686+0000 7fbe187898c0 1 > prepared operation key=XULE5KU5OITV tag=963KCXLUKKPP type=del size=1170117 2023-01-24T10:52:23.688 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.687+0000 7fbe187898c0 1 < completed write operation key=VBIA0USNT4OA tag=J98PH3LW25W8 type=write size=77793 2023-01-24T10:52:23.689 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.688+0000 7fbe187898c0 1 > prepared operation key=0VR1WPQLHOV0 tag=MTXARKFYAUB2 type=del size=3177114 2023-01-24T10:52:23.692 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.691+0000 7fbe187898c0 1 < suggested update operation key=SNBN0CZ6S9TA tag=TGAPUZ6Z234K type=write size=467403 2023-01-24T10:52:23.694 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.693+0000 7fbe187898c0 1 > prepared part key=_multipart_NU6FGGI1IUU7.2~V1ZOW7A8JPCT.0 size=1048576 2023-01-24T10:52:23.695 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.694+0000 7fbe187898c0 1 > prepared part key=_multipart_NU6FGGI1IUU7.2~V1ZOW7A8JPCT.1 size=490139 2023-01-24T10:52:23.696 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.694+0000 7fbe187898c0 1 > prepared multipart upload key=NU6FGGI1IUU7 upload=V1ZOW7A8JPCT tag=UQHV0FYR4KJH type=write size=2587291 2023-01-24T10:52:23.697 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.695+0000 7fbe187898c0 1 < completed part key=_multipart_K1AKVEEHDJEG.2~LAA3608930ZA.0 size=63218 2023-01-24T10:52:23.699 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.698+0000 7fbe187898c0 1 < completed multipart upload key=K1AKVEEHDJEG upload=LAA3608930ZA tag=YOJ6FY0QQSO7 type=write size=1111794 2023-01-24T10:52:23.701 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.700+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~0Z16J7TCUIP4.0 size=1048576 2023-01-24T10:52:23.703 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.701+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~0Z16J7TCUIP4.1 size=1029880 2023-01-24T10:52:23.703 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.701+0000 7fbe187898c0 1 > prepared multipart upload key=YGC2UEGMODS8 upload=0Z16J7TCUIP4 tag=O01DRT25MS8J type=write size=3127032 2023-01-24T10:52:23.704 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.703+0000 7fbe187898c0 1 < completed part key=_multipart_ZQCEEZ3RIC8F.2~D0CR6LC5R7ZS.0 size=535759 2023-01-24T10:52:23.710 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.709+0000 7fbe187898c0 1 < completed multipart upload key=ZQCEEZ3RIC8F upload=D0CR6LC5R7ZS tag=KANW300Q2D20 type=write size=1584335 2023-01-24T10:52:23.716 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.714+0000 7fbe187898c0 1 > prepared part key=_multipart_THP93R0E799L.2~J6UMYJQS0FDD.0 size=169073 2023-01-24T10:52:23.716 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.714+0000 7fbe187898c0 1 > prepared multipart upload key=THP93R0E799L upload=J6UMYJQS0FDD tag=PKYXDQO1NVAS type=write size=1217649 2023-01-24T10:52:23.718 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.717+0000 7fbe187898c0 1 < completed delete operation key=WDQNB5TEJUC6 tag=ETOKTDOQLFZ0 type=del 2023-01-24T10:52:23.724 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.723+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~NDD7GPB2S4BX.0 size=1048576 2023-01-24T10:52:23.727 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.725+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~NDD7GPB2S4BX.1 size=3957 2023-01-24T10:52:23.727 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.725+0000 7fbe187898c0 1 > prepared multipart upload key=YGC2UEGMODS8 upload=NDD7GPB2S4BX tag=3TV6AZXBVU6D type=write size=2101109 2023-01-24T10:52:23.728 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.727+0000 7fbe187898c0 1 < completed delete operation key=D0CGZ9OQK1AM tag=M1S1VMU0XEB1 type=del 2023-01-24T10:52:23.733 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.732+0000 7fbe187898c0 1 > prepared part key=_multipart_ZQCEEZ3RIC8F.2~GF5XM1196C2G.0 size=1048576 2023-01-24T10:52:23.737 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.736+0000 7fbe187898c0 1 > prepared part key=_multipart_ZQCEEZ3RIC8F.2~GF5XM1196C2G.1 size=373121 2023-01-24T10:52:23.738 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.736+0000 7fbe187898c0 1 > prepared multipart upload key=ZQCEEZ3RIC8F upload=GF5XM1196C2G tag=D71QACATXYL1 type=write size=2470273 2023-01-24T10:52:23.740 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.739+0000 7fbe187898c0 1 < suggested remove operation key=D92A7W1TH7X0 tag=H4ZK90Y4E363 type=write 2023-01-24T10:52:23.741 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.740+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=AVV428SY8V3O type=del size=616371 2023-01-24T10:52:23.743 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.741+0000 7fbe187898c0 1 < canceled operation key=1ZN0Y6AS7ISA tag=YWR368DB8L32 type=del size=2571958 2023-01-24T10:52:23.744 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.743+0000 7fbe187898c0 1 > prepared operation key=HJGFMANXEK4E tag=I41EZAG002DB type=del size=1727433 2023-01-24T10:52:23.745 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.744+0000 7fbe187898c0 1 < completed part key=_multipart_NU6FGGI1IUU7.2~I6QBJY8NXCFP.0 size=641001 2023-01-24T10:52:23.747 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.746+0000 7fbe187898c0 1 < completed multipart upload key=NU6FGGI1IUU7 upload=I6QBJY8NXCFP tag=C121TJQOT1SD type=write size=1689577 2023-01-24T10:52:23.749 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.747+0000 7fbe187898c0 1 > prepared operation key=Z4E99NU2FUTH tag=6JNFPWLM9CFO type=del size=2119513 2023-01-24T10:52:23.750 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.749+0000 7fbe187898c0 1 < completed delete operation key=THP93R0E799L tag=ZL5MENCZTI05 type=del 2023-01-24T10:52:23.752 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.751+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~WTZYCHF14AFZ.0 size=224379 2023-01-24T10:52:23.753 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.751+0000 7fbe187898c0 1 > prepared multipart upload key=9UV7IWZOV627 upload=WTZYCHF14AFZ tag=63HPA1FALPO8 type=write size=1272955 2023-01-24T10:52:23.754 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.752+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~V09EN41KOCZV.0 size=980116 2023-01-24T10:52:23.756 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.754+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=V09EN41KOCZV tag=HFDE598918PB type=write size=2028692 2023-01-24T10:52:23.758 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.757+0000 7fbe187898c0 1 > prepared part key=_multipart_D0CGZ9OQK1AM.2~LAF2MZLL6SVY.0 size=125401 2023-01-24T10:52:23.758 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.757+0000 7fbe187898c0 1 > prepared multipart upload key=D0CGZ9OQK1AM upload=LAF2MZLL6SVY tag=U3Q9G2MC2SAB type=write size=1173977 2023-01-24T10:52:23.759 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.757+0000 7fbe187898c0 1 < canceled part key=_multipart_C9J0MO49LDCS.2~CEEI7C21J28I.0 size=1048576 2023-01-24T10:52:23.760 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.758+0000 7fbe187898c0 1 < completed part key=_multipart_C9J0MO49LDCS.2~CEEI7C21J28I.1 size=1048576 2023-01-24T10:52:23.761 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.759+0000 7fbe187898c0 1 < completed part key=_multipart_C9J0MO49LDCS.2~CEEI7C21J28I.2 size=538261 2023-01-24T10:52:23.763 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.762+0000 7fbe187898c0 1 < canceled multipart upload key=C9J0MO49LDCS upload=CEEI7C21J28I tag=9U685Y5DBZF4 type=write size=3683989 2023-01-24T10:52:23.764 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.763+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=G9SKJIXIH81U type=del size=19991 2023-01-24T10:52:23.766 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.764+0000 7fbe187898c0 1 < completed delete operation key=A0DJSLNVH25F tag=QNQF4B09GAU6 type=del 2023-01-24T10:52:23.767 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.766+0000 7fbe187898c0 1 > prepared operation key=K1AKVEEHDJEG tag=RKZUQXSCQJA7 type=del size=3034698 2023-01-24T10:52:23.768 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.767+0000 7fbe187898c0 1 < completed delete operation key=C9J0MO49LDCS tag=L250DWRW2VFW type=del 2023-01-24T10:52:23.771 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.770+0000 7fbe187898c0 1 > prepared operation key=D92A7W1TH7X0 tag=1WWBDFIWRUWP type=del size=1408779 2023-01-24T10:52:23.774 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.773+0000 7fbe187898c0 1 < canceled operation key=ZMEBN0OQ1IPJ tag=DG5QK7LQDFP2 type=del size=2656609 2023-01-24T10:52:23.777 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.775+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=1I82XFSQMW8T type=del size=3214092 2023-01-24T10:52:23.779 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.778+0000 7fbe187898c0 1 < canceled operation key=XULE5KU5OITV tag=963KCXLUKKPP type=del size=1170117 2023-01-24T10:52:23.782 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.781+0000 7fbe187898c0 1 > prepared operation key=K1AKVEEHDJEG tag=AQG5K6RVFVAH type=del size=1411808 2023-01-24T10:52:23.787 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.786+0000 7fbe187898c0 1 < suggested remove operation key=0VR1WPQLHOV0 tag=MTXARKFYAUB2 type=del 2023-01-24T10:52:23.790 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.789+0000 7fbe187898c0 1 > prepared operation key=FDZVWZ0V33EB tag=FXIU4A7K66C3 type=del size=612487 2023-01-24T10:52:23.793 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.792+0000 7fbe187898c0 1 < completed part key=_multipart_NU6FGGI1IUU7.2~V1ZOW7A8JPCT.0 size=1048576 2023-01-24T10:52:23.794 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.793+0000 7fbe187898c0 1 < completed part key=_multipart_NU6FGGI1IUU7.2~V1ZOW7A8JPCT.1 size=490139 2023-01-24T10:52:23.796 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.795+0000 7fbe187898c0 1 < completed multipart upload key=NU6FGGI1IUU7 upload=V1ZOW7A8JPCT tag=UQHV0FYR4KJH type=write size=2587291 2023-01-24T10:52:23.798 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.796+0000 7fbe187898c0 1 > prepared operation key=FDZVWZ0V33EB tag=OA572UG341W0 type=del size=3312592 2023-01-24T10:52:23.799 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.797+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~0Z16J7TCUIP4.0 size=1048576 2023-01-24T10:52:23.800 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.799+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~0Z16J7TCUIP4.1 size=1029880 2023-01-24T10:52:23.802 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.801+0000 7fbe187898c0 1 < completed multipart upload key=YGC2UEGMODS8 upload=0Z16J7TCUIP4 tag=O01DRT25MS8J type=write size=3127032 2023-01-24T10:52:23.803 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.802+0000 7fbe187898c0 1 > prepared operation key=Z4E99NU2FUTH tag=7IV5IEX6BZ1X type=del size=323715 2023-01-24T10:52:23.805 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.803+0000 7fbe187898c0 1 < completed part key=_multipart_THP93R0E799L.2~J6UMYJQS0FDD.0 size=169073 2023-01-24T10:52:23.807 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.805+0000 7fbe187898c0 1 < completed multipart upload key=THP93R0E799L upload=J6UMYJQS0FDD tag=PKYXDQO1NVAS type=write size=1217649 2023-01-24T10:52:23.808 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.807+0000 7fbe187898c0 1 > prepared operation key=Z8G7Y4RGVWR8 tag=3UVOV938QE6U type=write size=747671 2023-01-24T10:52:23.809 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.808+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~NDD7GPB2S4BX.0 size=1048576 2023-01-24T10:52:23.811 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.809+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~NDD7GPB2S4BX.1 size=3957 2023-01-24T10:52:23.813 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.812+0000 7fbe187898c0 1 < completed multipart upload key=YGC2UEGMODS8 upload=NDD7GPB2S4BX tag=3TV6AZXBVU6D type=write size=2101109 2023-01-24T10:52:23.815 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.814+0000 7fbe187898c0 1 > prepared part key=_multipart_Z4E99NU2FUTH.2~XLWOKH9F4941.0 size=1048576 2023-01-24T10:52:23.816 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.815+0000 7fbe187898c0 1 > prepared part key=_multipart_Z4E99NU2FUTH.2~XLWOKH9F4941.1 size=962657 2023-01-24T10:52:23.816 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.815+0000 7fbe187898c0 1 > prepared multipart upload key=Z4E99NU2FUTH upload=XLWOKH9F4941 tag=2WM9SQNWQZUJ type=write size=3059809 2023-01-24T10:52:23.817 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.815+0000 7fbe187898c0 1 < canceled part key=_multipart_ZQCEEZ3RIC8F.2~GF5XM1196C2G.0 size=1048576 2023-01-24T10:52:23.817 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.816+0000 7fbe187898c0 1 < completed part key=_multipart_ZQCEEZ3RIC8F.2~GF5XM1196C2G.1 size=373121 2023-01-24T10:52:23.820 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.818+0000 7fbe187898c0 1 < canceled multipart upload key=ZQCEEZ3RIC8F upload=GF5XM1196C2G tag=D71QACATXYL1 type=write size=2470273 2023-01-24T10:52:23.821 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.820+0000 7fbe187898c0 1 > prepared operation key=YTEKYBINDI7I tag=WN3FKILO533U type=del size=3322588 2023-01-24T10:52:23.822 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.821+0000 7fbe187898c0 1 < completed delete operation key=MQAFAWM6WGH1 tag=AVV428SY8V3O type=del 2023-01-24T10:52:23.824 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.822+0000 7fbe187898c0 1 > prepared operation key=HJGFMANXEK4E tag=YM79FW583V82 type=del size=1388193 2023-01-24T10:52:23.825 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.824+0000 7fbe187898c0 1 < completed delete operation key=HJGFMANXEK4E tag=I41EZAG002DB type=del 2023-01-24T10:52:23.826 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.825+0000 7fbe187898c0 1 > prepared operation key=D0CGZ9OQK1AM tag=PJNO2I88FDI6 type=del size=355753 2023-01-24T10:52:23.828 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.826+0000 7fbe187898c0 1 < completed delete operation key=Z4E99NU2FUTH tag=6JNFPWLM9CFO type=del 2023-01-24T10:52:23.830 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.829+0000 7fbe187898c0 1 > prepared part key=_multipart_SNBN0CZ6S9TA.2~LJMVGVUAGJ92.0 size=1048576 2023-01-24T10:52:23.831 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.830+0000 7fbe187898c0 1 > prepared part key=_multipart_SNBN0CZ6S9TA.2~LJMVGVUAGJ92.1 size=836959 2023-01-24T10:52:23.831 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.830+0000 7fbe187898c0 1 > prepared multipart upload key=SNBN0CZ6S9TA upload=LJMVGVUAGJ92 tag=CIT6GWP3C6CF type=write size=2934111 2023-01-24T10:52:23.832 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.831+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~WTZYCHF14AFZ.0 size=224379 2023-01-24T10:52:23.836 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.835+0000 7fbe187898c0 1 < completed multipart upload key=9UV7IWZOV627 upload=WTZYCHF14AFZ tag=63HPA1FALPO8 type=write size=1272955 2023-01-24T10:52:23.839 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.837+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=KHF9A0EP6H13 type=del size=2014101 2023-01-24T10:52:23.840 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.839+0000 7fbe187898c0 1 < completed part key=_multipart_D0CGZ9OQK1AM.2~LAF2MZLL6SVY.0 size=125401 2023-01-24T10:52:23.846 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.844+0000 7fbe187898c0 1 < completed multipart upload key=D0CGZ9OQK1AM upload=LAF2MZLL6SVY tag=U3Q9G2MC2SAB type=write size=1173977 2023-01-24T10:52:23.847 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.846+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=FEVU39G789HR type=del size=4069098 2023-01-24T10:52:23.848 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.847+0000 7fbe187898c0 1 < completed delete operation key=NU6FGGI1IUU7 tag=G9SKJIXIH81U type=del 2023-01-24T10:52:23.850 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.848+0000 7fbe187898c0 1 > prepared operation key=Z8G7Y4RGVWR8 tag=5GXMXSCVSRJD type=del size=1329997 2023-01-24T10:52:23.851 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.850+0000 7fbe187898c0 1 < canceled operation key=K1AKVEEHDJEG tag=RKZUQXSCQJA7 type=del size=3034698 2023-01-24T10:52:23.853 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.852+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~U3BDDILRCXJ8.0 size=1048576 2023-01-24T10:52:23.855 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.853+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~U3BDDILRCXJ8.1 size=1048576 2023-01-24T10:52:23.856 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.855+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~U3BDDILRCXJ8.2 size=974922 2023-01-24T10:52:23.857 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.855+0000 7fbe187898c0 1 > prepared multipart upload key=MQAFAWM6WGH1 upload=U3BDDILRCXJ8 tag=1FX1NZE6MCDE type=write size=4120650 2023-01-24T10:52:23.857 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.856+0000 7fbe187898c0 1 < completed delete operation key=D92A7W1TH7X0 tag=1WWBDFIWRUWP type=del 2023-01-24T10:52:23.860 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.859+0000 7fbe187898c0 1 > prepared part key=_multipart_THP93R0E799L.2~MAM6DKD9CM0C.0 size=967929 2023-01-24T10:52:23.860 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.859+0000 7fbe187898c0 1 > prepared multipart upload key=THP93R0E799L upload=MAM6DKD9CM0C tag=4B0EQ4UP0RCS type=write size=2016505 2023-01-24T10:52:23.861 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.860+0000 7fbe187898c0 1 < completed delete operation key=YGC2UEGMODS8 tag=1I82XFSQMW8T type=del 2023-01-24T10:52:23.864 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.863+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~TAGNVKWC7VSU.0 size=1048576 2023-01-24T10:52:23.865 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.864+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~TAGNVKWC7VSU.1 size=305369 2023-01-24T10:52:23.865 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.864+0000 7fbe187898c0 1 > prepared multipart upload key=MQAFAWM6WGH1 upload=TAGNVKWC7VSU tag=3P03UH3WNJ7T type=write size=2402521 2023-01-24T10:52:23.866 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.865+0000 7fbe187898c0 1 < completed delete operation key=K1AKVEEHDJEG tag=AQG5K6RVFVAH type=del 2023-01-24T10:52:23.868 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.866+0000 7fbe187898c0 1 > prepared operation key=K1AKVEEHDJEG tag=43H8HZNM16HP type=del size=1684637 2023-01-24T10:52:23.869 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.868+0000 7fbe187898c0 1 < completed delete operation key=FDZVWZ0V33EB tag=FXIU4A7K66C3 type=del 2023-01-24T10:52:23.872 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.870+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~2V01H0DAJBE3.0 size=1048576 2023-01-24T10:52:23.873 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.871+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~2V01H0DAJBE3.1 size=1048576 2023-01-24T10:52:23.874 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.872+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~2V01H0DAJBE3.2 size=500725 2023-01-24T10:52:23.874 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.872+0000 7fbe187898c0 1 > prepared multipart upload key=Z5DPD9MNHN2H upload=2V01H0DAJBE3 tag=95MYA3ACLXRQ type=write size=3646453 2023-01-24T10:52:23.876 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.875+0000 7fbe187898c0 1 < suggested update operation key=FDZVWZ0V33EB tag=OA572UG341W0 type=del size=3312592 2023-01-24T10:52:23.877 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.876+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=U2JQPTN71OSH type=del size=1372097 2023-01-24T10:52:23.879 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.877+0000 7fbe187898c0 1 < completed delete operation key=Z4E99NU2FUTH tag=7IV5IEX6BZ1X type=del 2023-01-24T10:52:23.884 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.882+0000 7fbe187898c0 1 > prepared part key=_multipart_K1AKVEEHDJEG.2~2OQGQ8NDQ5LE.0 size=1048576 2023-01-24T10:52:23.886 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.885+0000 7fbe187898c0 1 > prepared part key=_multipart_K1AKVEEHDJEG.2~2OQGQ8NDQ5LE.1 size=483727 2023-01-24T10:52:23.887 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.885+0000 7fbe187898c0 1 > prepared multipart upload key=K1AKVEEHDJEG upload=2OQGQ8NDQ5LE tag=HQ6DQ0KI25PT type=write size=2580879 2023-01-24T10:52:23.889 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.888+0000 7fbe187898c0 1 < completed write operation key=Z8G7Y4RGVWR8 tag=3UVOV938QE6U type=write size=747671 2023-01-24T10:52:23.891 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.889+0000 7fbe187898c0 1 > prepared operation key=1ZN0Y6AS7ISA tag=PSB4FIDLH5HJ type=del size=4034169 2023-01-24T10:52:23.892 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.891+0000 7fbe187898c0 1 < completed part key=_multipart_Z4E99NU2FUTH.2~XLWOKH9F4941.0 size=1048576 2023-01-24T10:52:23.893 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.892+0000 7fbe187898c0 1 < completed part key=_multipart_Z4E99NU2FUTH.2~XLWOKH9F4941.1 size=962657 2023-01-24T10:52:23.895 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.894+0000 7fbe187898c0 1 < completed multipart upload key=Z4E99NU2FUTH upload=XLWOKH9F4941 tag=2WM9SQNWQZUJ type=write size=3059809 2023-01-24T10:52:23.896 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.895+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=R1EK6SN0HCQ7 type=del size=873362 2023-01-24T10:52:23.898 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.896+0000 7fbe187898c0 1 < completed delete operation key=YTEKYBINDI7I tag=WN3FKILO533U type=del 2023-01-24T10:52:23.899 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.898+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=2PPB27MDM10G type=del size=918007 2023-01-24T10:52:23.900 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.899+0000 7fbe187898c0 1 < completed delete operation key=HJGFMANXEK4E tag=YM79FW583V82 type=del 2023-01-24T10:52:23.901 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.900+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=XES3CYZWII3D type=write size=706064 2023-01-24T10:52:23.903 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.901+0000 7fbe187898c0 1 < completed delete operation key=D0CGZ9OQK1AM tag=PJNO2I88FDI6 type=del 2023-01-24T10:52:23.905 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.904+0000 7fbe187898c0 1 > prepared part key=_multipart_SNBN0CZ6S9TA.2~RD3N1VBT09BL.0 size=556024 2023-01-24T10:52:23.905 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.904+0000 7fbe187898c0 1 > prepared multipart upload key=SNBN0CZ6S9TA upload=RD3N1VBT09BL tag=9ONUJT001H0W type=write size=1604600 2023-01-24T10:52:23.906 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.905+0000 7fbe187898c0 1 < completed part key=_multipart_SNBN0CZ6S9TA.2~LJMVGVUAGJ92.0 size=1048576 2023-01-24T10:52:23.907 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.906+0000 7fbe187898c0 1 < completed part key=_multipart_SNBN0CZ6S9TA.2~LJMVGVUAGJ92.1 size=836959 2023-01-24T10:52:23.910 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.908+0000 7fbe187898c0 1 < completed multipart upload key=SNBN0CZ6S9TA upload=LJMVGVUAGJ92 tag=CIT6GWP3C6CF type=write size=2934111 2023-01-24T10:52:23.911 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.910+0000 7fbe187898c0 1 > prepared operation key=SNBN0CZ6S9TA tag=FVMMYDXLT0FB type=del size=1318375 2023-01-24T10:52:23.912 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.911+0000 7fbe187898c0 1 < completed delete operation key=WDQNB5TEJUC6 tag=KHF9A0EP6H13 type=del 2023-01-24T10:52:23.914 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.913+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~KZN1IPGB47M4.0 size=1048576 2023-01-24T10:52:23.915 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.914+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~KZN1IPGB47M4.1 size=1048576 2023-01-24T10:52:23.917 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.915+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~KZN1IPGB47M4.2 size=383927 2023-01-24T10:52:23.917 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.915+0000 7fbe187898c0 1 > prepared multipart upload key=5W2UE0AZTOM0 upload=KZN1IPGB47M4 tag=F7XJX2PU5POZ type=write size=3529655 2023-01-24T10:52:23.918 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.917+0000 7fbe187898c0 1 < canceled operation key=THP93R0E799L tag=FEVU39G789HR type=del size=4069098 2023-01-24T10:52:23.919 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.918+0000 7fbe187898c0 1 > prepared operation key=D0CGZ9OQK1AM tag=KNI620BJQFV3 type=del size=3877265 2023-01-24T10:52:23.922 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.921+0000 7fbe187898c0 1 < suggested remove operation key=Z8G7Y4RGVWR8 tag=5GXMXSCVSRJD type=del 2023-01-24T10:52:23.925 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.924+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~MVCTP6Z4FAN0.0 size=440625 2023-01-24T10:52:23.926 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.924+0000 7fbe187898c0 1 > prepared multipart upload key=3GJVNT6TXTK6 upload=MVCTP6Z4FAN0 tag=QX7YWW9ZU0FB type=write size=1489201 2023-01-24T10:52:23.927 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.925+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~U3BDDILRCXJ8.0 size=1048576 2023-01-24T10:52:23.928 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.926+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~U3BDDILRCXJ8.1 size=1048576 2023-01-24T10:52:23.929 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.927+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~U3BDDILRCXJ8.2 size=974922 2023-01-24T10:52:23.931 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.930+0000 7fbe187898c0 1 < completed multipart upload key=MQAFAWM6WGH1 upload=U3BDDILRCXJ8 tag=1FX1NZE6MCDE type=write size=4120650 2023-01-24T10:52:23.933 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.932+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=RM8KETBQKK9S type=write size=695968 2023-01-24T10:52:23.934 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.933+0000 7fbe187898c0 1 < completed part key=_multipart_THP93R0E799L.2~MAM6DKD9CM0C.0 size=967929 2023-01-24T10:52:23.936 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.935+0000 7fbe187898c0 1 < completed multipart upload key=THP93R0E799L upload=MAM6DKD9CM0C tag=4B0EQ4UP0RCS type=write size=2016505 2023-01-24T10:52:23.939 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.938+0000 7fbe187898c0 1 > prepared part key=_multipart_SNBN0CZ6S9TA.2~HY66I5FNEFA7.0 size=617205 2023-01-24T10:52:23.939 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.938+0000 7fbe187898c0 1 > prepared multipart upload key=SNBN0CZ6S9TA upload=HY66I5FNEFA7 tag=D96QF0L61KH2 type=write size=1665781 2023-01-24T10:52:23.941 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.939+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~TAGNVKWC7VSU.0 size=1048576 2023-01-24T10:52:23.941 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.940+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~TAGNVKWC7VSU.1 size=305369 2023-01-24T10:52:23.944 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.943+0000 7fbe187898c0 1 < completed multipart upload key=MQAFAWM6WGH1 upload=TAGNVKWC7VSU tag=3P03UH3WNJ7T type=write size=2402521 2023-01-24T10:52:23.949 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.948+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~CNCLUNZ297UU.0 size=1048576 2023-01-24T10:52:23.950 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.949+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~CNCLUNZ297UU.1 size=543121 2023-01-24T10:52:23.951 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.949+0000 7fbe187898c0 1 > prepared multipart upload key=YGC2UEGMODS8 upload=CNCLUNZ297UU tag=6BMBAYRFHT9B type=write size=2640273 2023-01-24T10:52:23.953 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.951+0000 7fbe187898c0 1 < suggested update operation key=K1AKVEEHDJEG tag=43H8HZNM16HP type=del size=1684637 2023-01-24T10:52:23.955 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.954+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~TWT2AWVZVQZV.0 size=1048576 2023-01-24T10:52:23.956 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.955+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~TWT2AWVZVQZV.1 size=1048576 2023-01-24T10:52:23.957 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.956+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~TWT2AWVZVQZV.2 size=47832 2023-01-24T10:52:23.957 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.956+0000 7fbe187898c0 1 > prepared multipart upload key=5W2UE0AZTOM0 upload=TWT2AWVZVQZV tag=M2TT67483WEP type=write size=3193560 2023-01-24T10:52:23.959 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.957+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~2V01H0DAJBE3.0 size=1048576 2023-01-24T10:52:23.960 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.958+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~2V01H0DAJBE3.1 size=1048576 2023-01-24T10:52:23.960 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.959+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~2V01H0DAJBE3.2 size=500725 2023-01-24T10:52:23.963 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.961+0000 7fbe187898c0 1 < completed multipart upload key=Z5DPD9MNHN2H upload=2V01H0DAJBE3 tag=95MYA3ACLXRQ type=write size=3646453 2023-01-24T10:52:23.964 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.963+0000 7fbe187898c0 1 > prepared operation key=RV7FCL2UARTQ tag=WKXY5UFZH8AG type=write size=220236 2023-01-24T10:52:23.965 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.964+0000 7fbe187898c0 1 < completed delete operation key=WDQNB5TEJUC6 tag=U2JQPTN71OSH type=del 2023-01-24T10:52:23.966 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.965+0000 7fbe187898c0 1 > prepared operation key=5W2UE0AZTOM0 tag=VG8XJ7B6LDSK type=del size=2485493 2023-01-24T10:52:23.969 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.968+0000 7fbe187898c0 1 < completed part key=_multipart_K1AKVEEHDJEG.2~2OQGQ8NDQ5LE.0 size=1048576 2023-01-24T10:52:23.970 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.968+0000 7fbe187898c0 1 < canceled part key=_multipart_K1AKVEEHDJEG.2~2OQGQ8NDQ5LE.1 size=483727 2023-01-24T10:52:23.974 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.973+0000 7fbe187898c0 1 < canceled multipart upload key=K1AKVEEHDJEG upload=2OQGQ8NDQ5LE tag=HQ6DQ0KI25PT type=write size=2580879 2023-01-24T10:52:23.980 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.978+0000 7fbe187898c0 1 > prepared part key=_multipart_A0DJSLNVH25F.2~9364OAW2MOH3.0 size=1048576 2023-01-24T10:52:23.985 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.981+0000 7fbe187898c0 1 > prepared part key=_multipart_A0DJSLNVH25F.2~9364OAW2MOH3.1 size=558580 2023-01-24T10:52:23.985 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.981+0000 7fbe187898c0 1 > prepared multipart upload key=A0DJSLNVH25F upload=9364OAW2MOH3 tag=7Q0UO1S2J1U3 type=write size=2655732 2023-01-24T10:52:23.986 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.984+0000 7fbe187898c0 1 < completed delete operation key=1ZN0Y6AS7ISA tag=PSB4FIDLH5HJ type=del 2023-01-24T10:52:23.989 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.988+0000 7fbe187898c0 1 > prepared part key=_multipart_YTEKYBINDI7I.2~8VK0UN0N2J3I.0 size=1048576 2023-01-24T10:52:23.991 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.990+0000 7fbe187898c0 1 > prepared part key=_multipart_YTEKYBINDI7I.2~8VK0UN0N2J3I.1 size=476889 2023-01-24T10:52:23.992 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.990+0000 7fbe187898c0 1 > prepared multipart upload key=YTEKYBINDI7I upload=8VK0UN0N2J3I tag=2YDFLE75WJ85 type=write size=2574041 2023-01-24T10:52:23.994 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.993+0000 7fbe187898c0 1 < suggested remove operation key=WDQNB5TEJUC6 tag=R1EK6SN0HCQ7 type=del 2023-01-24T10:52:23.995 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.994+0000 7fbe187898c0 1 > prepared operation key=D92A7W1TH7X0 tag=42H2WGKKC8VQ type=del size=3640351 2023-01-24T10:52:23.996 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.995+0000 7fbe187898c0 1 < completed delete operation key=MQAFAWM6WGH1 tag=2PPB27MDM10G type=del 2023-01-24T10:52:23.998 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:23.997+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~002ITLHXYM1I.0 size=1048576 2023-01-24T10:52:24.001 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.000+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~002ITLHXYM1I.1 size=325045 2023-01-24T10:52:24.002 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.000+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=002ITLHXYM1I tag=UV9GMGUJGOJZ type=write size=2422197 2023-01-24T10:52:24.004 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.003+0000 7fbe187898c0 1 < canceled operation key=NU6FGGI1IUU7 tag=XES3CYZWII3D type=write size=706064 2023-01-24T10:52:24.006 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.005+0000 7fbe187898c0 1 > prepared operation key=SNBN0CZ6S9TA tag=DZKAF7SGJTJI type=write size=701934 2023-01-24T10:52:24.008 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.007+0000 7fbe187898c0 1 < completed part key=_multipart_SNBN0CZ6S9TA.2~RD3N1VBT09BL.0 size=556024 2023-01-24T10:52:24.014 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.012+0000 7fbe187898c0 1 < completed multipart upload key=SNBN0CZ6S9TA upload=RD3N1VBT09BL tag=9ONUJT001H0W type=write size=1604600 2023-01-24T10:52:24.019 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.018+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~3G0PQRPB28ET.0 size=884309 2023-01-24T10:52:24.020 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.018+0000 7fbe187898c0 1 > prepared multipart upload key=MQAFAWM6WGH1 upload=3G0PQRPB28ET tag=90630FXGGNO0 type=write size=1932885 2023-01-24T10:52:24.022 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.020+0000 7fbe187898c0 1 < suggested update operation key=SNBN0CZ6S9TA tag=FVMMYDXLT0FB type=del size=1318375 2023-01-24T10:52:24.024 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.022+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=4SM31IAI0UCU type=write size=374276 2023-01-24T10:52:24.025 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.024+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~KZN1IPGB47M4.0 size=1048576 2023-01-24T10:52:24.026 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.025+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~KZN1IPGB47M4.1 size=1048576 2023-01-24T10:52:24.027 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.026+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~KZN1IPGB47M4.2 size=383927 2023-01-24T10:52:24.029 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.028+0000 7fbe187898c0 1 < completed multipart upload key=5W2UE0AZTOM0 upload=KZN1IPGB47M4 tag=F7XJX2PU5POZ type=write size=3529655 2023-01-24T10:52:24.031 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.030+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~EA41A2VJ9NXU.0 size=368936 2023-01-24T10:52:24.032 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.030+0000 7fbe187898c0 1 > prepared multipart upload key=3GJVNT6TXTK6 upload=EA41A2VJ9NXU tag=E3EWML6OUSEE type=write size=1417512 2023-01-24T10:52:24.033 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.031+0000 7fbe187898c0 1 < canceled operation key=D0CGZ9OQK1AM tag=KNI620BJQFV3 type=del size=3877265 2023-01-24T10:52:24.038 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.037+0000 7fbe187898c0 1 > prepared part key=_multipart_K1AKVEEHDJEG.2~ON91ZFX7GWU5.0 size=1048576 2023-01-24T10:52:24.041 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.039+0000 7fbe187898c0 1 > prepared part key=_multipart_K1AKVEEHDJEG.2~ON91ZFX7GWU5.1 size=463139 2023-01-24T10:52:24.041 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.039+0000 7fbe187898c0 1 > prepared multipart upload key=K1AKVEEHDJEG upload=ON91ZFX7GWU5 tag=21IYEX00GU62 type=write size=2560291 2023-01-24T10:52:24.043 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.042+0000 7fbe187898c0 1 < completed part key=_multipart_3GJVNT6TXTK6.2~MVCTP6Z4FAN0.0 size=440625 2023-01-24T10:52:24.049 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.048+0000 7fbe187898c0 1 < completed multipart upload key=3GJVNT6TXTK6 upload=MVCTP6Z4FAN0 tag=QX7YWW9ZU0FB type=write size=1489201 2023-01-24T10:52:24.051 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.049+0000 7fbe187898c0 1 > prepared operation key=S7XZ2Q32GCMK tag=LJ2KOOG6IV7P type=del size=3417118 2023-01-24T10:52:24.055 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.053+0000 7fbe187898c0 1 < completed write operation key=NU6FGGI1IUU7 tag=RM8KETBQKK9S type=write size=695968 2023-01-24T10:52:24.057 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.056+0000 7fbe187898c0 1 > prepared operation key=TIE8E7PAI3NC tag=GW5KS4BBVNYT type=del size=1887286 2023-01-24T10:52:24.060 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.058+0000 7fbe187898c0 1 < completed part key=_multipart_SNBN0CZ6S9TA.2~HY66I5FNEFA7.0 size=617205 2023-01-24T10:52:24.062 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.061+0000 7fbe187898c0 1 < completed multipart upload key=SNBN0CZ6S9TA upload=HY66I5FNEFA7 tag=D96QF0L61KH2 type=write size=1665781 2023-01-24T10:52:24.063 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.062+0000 7fbe187898c0 1 > prepared operation key=SNBN0CZ6S9TA tag=VNERRHHCDR34 type=write size=363937 2023-01-24T10:52:24.064 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.063+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~CNCLUNZ297UU.0 size=1048576 2023-01-24T10:52:24.065 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.064+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~CNCLUNZ297UU.1 size=543121 2023-01-24T10:52:24.068 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.066+0000 7fbe187898c0 1 < completed multipart upload key=YGC2UEGMODS8 upload=CNCLUNZ297UU tag=6BMBAYRFHT9B type=write size=2640273 2023-01-24T10:52:24.070 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.068+0000 7fbe187898c0 1 > prepared part key=_multipart_HJGFMANXEK4E.2~OTP190PXZ8F2.0 size=578231 2023-01-24T10:52:24.070 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.068+0000 7fbe187898c0 1 > prepared multipart upload key=HJGFMANXEK4E upload=OTP190PXZ8F2 tag=RRUCIOPGXI04 type=write size=1626807 2023-01-24T10:52:24.070 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.069+0000 7fbe187898c0 1 < canceled part key=_multipart_5W2UE0AZTOM0.2~TWT2AWVZVQZV.0 size=1048576 2023-01-24T10:52:24.071 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.069+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~TWT2AWVZVQZV.1 size=1048576 2023-01-24T10:52:24.072 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.070+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~TWT2AWVZVQZV.2 size=47832 2023-01-24T10:52:24.074 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.073+0000 7fbe187898c0 1 < canceled multipart upload key=5W2UE0AZTOM0 upload=TWT2AWVZVQZV tag=M2TT67483WEP type=write size=3193560 2023-01-24T10:52:24.075 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.074+0000 7fbe187898c0 1 > prepared operation key=YTEKYBINDI7I tag=XHTMSWTN19W9 type=del size=3930576 2023-01-24T10:52:24.076 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.075+0000 7fbe187898c0 1 < completed write operation key=RV7FCL2UARTQ tag=WKXY5UFZH8AG type=write size=220236 2023-01-24T10:52:24.078 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.076+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=S11MJVWVJBR1 type=del size=1704317 2023-01-24T10:52:24.079 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.078+0000 7fbe187898c0 1 < canceled operation key=5W2UE0AZTOM0 tag=VG8XJ7B6LDSK type=del size=2485493 2023-01-24T10:52:24.080 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.079+0000 7fbe187898c0 1 > prepared operation key=A0DJSLNVH25F tag=55Z4ZWWVDBSX type=del size=40660 2023-01-24T10:52:24.081 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.080+0000 7fbe187898c0 1 < completed part key=_multipart_A0DJSLNVH25F.2~9364OAW2MOH3.0 size=1048576 2023-01-24T10:52:24.082 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.081+0000 7fbe187898c0 1 < completed part key=_multipart_A0DJSLNVH25F.2~9364OAW2MOH3.1 size=558580 2023-01-24T10:52:24.085 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.083+0000 7fbe187898c0 1 < completed multipart upload key=A0DJSLNVH25F upload=9364OAW2MOH3 tag=7Q0UO1S2J1U3 type=write size=2655732 2023-01-24T10:52:24.087 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.086+0000 7fbe187898c0 1 > prepared part key=_multipart_Z4E99NU2FUTH.2~OQVVUCL89FM2.0 size=1048576 2023-01-24T10:52:24.088 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.087+0000 7fbe187898c0 1 > prepared part key=_multipart_Z4E99NU2FUTH.2~OQVVUCL89FM2.1 size=594403 2023-01-24T10:52:24.088 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.087+0000 7fbe187898c0 1 > prepared multipart upload key=Z4E99NU2FUTH upload=OQVVUCL89FM2 tag=C1W5Y9IZK4HB type=write size=2691555 2023-01-24T10:52:24.093 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.088+0000 7fbe187898c0 1 < completed part key=_multipart_YTEKYBINDI7I.2~8VK0UN0N2J3I.0 size=1048576 2023-01-24T10:52:24.093 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.089+0000 7fbe187898c0 1 < completed part key=_multipart_YTEKYBINDI7I.2~8VK0UN0N2J3I.1 size=476889 2023-01-24T10:52:24.094 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.091+0000 7fbe187898c0 1 < completed multipart upload key=YTEKYBINDI7I upload=8VK0UN0N2J3I tag=2YDFLE75WJ85 type=write size=2574041 2023-01-24T10:52:24.094 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.092+0000 7fbe187898c0 1 > prepared operation key=5W2UE0AZTOM0 tag=3L3XQ8QNJNXA type=del size=3229600 2023-01-24T10:52:24.095 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.094+0000 7fbe187898c0 1 < canceled operation key=D92A7W1TH7X0 tag=42H2WGKKC8VQ type=del size=3640351 2023-01-24T10:52:24.096 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.095+0000 7fbe187898c0 1 > prepared operation key=RV7FCL2UARTQ tag=LK3PMBD9T418 type=del size=3057215 2023-01-24T10:52:24.097 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.096+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~002ITLHXYM1I.0 size=1048576 2023-01-24T10:52:24.099 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.098+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~002ITLHXYM1I.1 size=325045 2023-01-24T10:52:24.103 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.101+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=002ITLHXYM1I tag=UV9GMGUJGOJZ type=write size=2422197 2023-01-24T10:52:24.105 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.104+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~Q7HAQ2I0LXE2.0 size=550334 2023-01-24T10:52:24.106 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.104+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=Q7HAQ2I0LXE2 tag=JYTWEUDJA436 type=write size=1598910 2023-01-24T10:52:24.106 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.105+0000 7fbe187898c0 -1 < failed to complete operation key=SNBN0CZ6S9TA tag=DZKAF7SGJTJI type=write size=701934: (22) Invalid argument 2023-01-24T10:52:24.109 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.107+0000 7fbe187898c0 1 > prepared part key=_multipart_D92A7W1TH7X0.2~NOW8SVL01LER.0 size=241693 2023-01-24T10:52:24.109 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.107+0000 7fbe187898c0 1 > prepared multipart upload key=D92A7W1TH7X0 upload=NOW8SVL01LER tag=GYTRGRRY2K99 type=write size=1290269 2023-01-24T10:52:24.110 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.109+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~3G0PQRPB28ET.0 size=884309 2023-01-24T10:52:24.112 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.110+0000 7fbe187898c0 1 < completed multipart upload key=MQAFAWM6WGH1 upload=3G0PQRPB28ET tag=90630FXGGNO0 type=write size=1932885 2023-01-24T10:52:24.113 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.112+0000 7fbe187898c0 1 > prepared operation key=5W2UE0AZTOM0 tag=6LOW9OKY2PGI type=del size=842019 2023-01-24T10:52:24.114 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.113+0000 7fbe187898c0 1 < completed write operation key=THP93R0E799L tag=4SM31IAI0UCU type=write size=374276 2023-01-24T10:52:24.116 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.115+0000 7fbe187898c0 1 > prepared part key=_multipart_S7XZ2Q32GCMK.2~7YLM1NHMXHJ3.0 size=1048576 2023-01-24T10:52:24.117 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.116+0000 7fbe187898c0 1 > prepared part key=_multipart_S7XZ2Q32GCMK.2~7YLM1NHMXHJ3.1 size=1048576 2023-01-24T10:52:24.118 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.117+0000 7fbe187898c0 1 > prepared part key=_multipart_S7XZ2Q32GCMK.2~7YLM1NHMXHJ3.2 size=630495 2023-01-24T10:52:24.119 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.117+0000 7fbe187898c0 1 > prepared multipart upload key=S7XZ2Q32GCMK upload=7YLM1NHMXHJ3 tag=Y6JUAOT6NNU5 type=write size=3776223 2023-01-24T10:52:24.120 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.118+0000 7fbe187898c0 1 < completed part key=_multipart_3GJVNT6TXTK6.2~EA41A2VJ9NXU.0 size=368936 2023-01-24T10:52:24.122 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.120+0000 7fbe187898c0 1 < completed multipart upload key=3GJVNT6TXTK6 upload=EA41A2VJ9NXU tag=E3EWML6OUSEE type=write size=1417512 2023-01-24T10:52:24.124 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.123+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~A991IBGRKV72.0 size=1048576 2023-01-24T10:52:24.125 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.124+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~A991IBGRKV72.1 size=357774 2023-01-24T10:52:24.125 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.124+0000 7fbe187898c0 1 > prepared multipart upload key=5W2UE0AZTOM0 upload=A991IBGRKV72 tag=9KODTIIINRXZ type=write size=2454926 2023-01-24T10:52:24.126 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.125+0000 7fbe187898c0 1 < completed part key=_multipart_K1AKVEEHDJEG.2~ON91ZFX7GWU5.0 size=1048576 2023-01-24T10:52:24.127 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.126+0000 7fbe187898c0 1 < completed part key=_multipart_K1AKVEEHDJEG.2~ON91ZFX7GWU5.1 size=463139 2023-01-24T10:52:24.130 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.128+0000 7fbe187898c0 1 < completed multipart upload key=K1AKVEEHDJEG upload=ON91ZFX7GWU5 tag=21IYEX00GU62 type=write size=2560291 2023-01-24T10:52:24.132 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.131+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~3L4JL16IDC8I.0 size=1048576 2023-01-24T10:52:24.133 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.132+0000 7fbe187898c0 1 > prepared part key=_multipart_FDZVWZ0V33EB.2~3L4JL16IDC8I.1 size=372073 2023-01-24T10:52:24.133 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.132+0000 7fbe187898c0 1 > prepared multipart upload key=FDZVWZ0V33EB upload=3L4JL16IDC8I tag=XG3TNCYSEGFZ type=write size=2469225 2023-01-24T10:52:24.139 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.137+0000 7fbe187898c0 1 < suggested update operation key=S7XZ2Q32GCMK tag=LJ2KOOG6IV7P type=del size=3417118 2023-01-24T10:52:24.144 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.143+0000 7fbe187898c0 1 > prepared part key=_multipart_ZMEBN0OQ1IPJ.2~H37X886SQVB3.0 size=1048576 2023-01-24T10:52:24.147 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.146+0000 7fbe187898c0 1 > prepared part key=_multipart_ZMEBN0OQ1IPJ.2~H37X886SQVB3.1 size=1048576 2023-01-24T10:52:24.150 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.148+0000 7fbe187898c0 1 > prepared part key=_multipart_ZMEBN0OQ1IPJ.2~H37X886SQVB3.2 size=833483 2023-01-24T10:52:24.150 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.148+0000 7fbe187898c0 1 > prepared multipart upload key=ZMEBN0OQ1IPJ upload=H37X886SQVB3 tag=O70MATOM1Y0O type=write size=3979211 2023-01-24T10:52:24.153 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.151+0000 7fbe187898c0 1 < completed delete operation key=TIE8E7PAI3NC tag=GW5KS4BBVNYT type=del 2023-01-24T10:52:24.156 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.154+0000 7fbe187898c0 1 > prepared operation key=Z8G7Y4RGVWR8 tag=SEJOT8N3TZIU type=del size=3058914 2023-01-24T10:52:24.158 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.157+0000 7fbe187898c0 1 < completed write operation key=SNBN0CZ6S9TA tag=VNERRHHCDR34 type=write size=363937 2023-01-24T10:52:24.161 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.160+0000 7fbe187898c0 1 > prepared operation key=63PDRJCRVEI1 tag=9R8CWWR818TP type=del size=2565198 2023-01-24T10:52:24.162 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.161+0000 7fbe187898c0 1 < completed part key=_multipart_HJGFMANXEK4E.2~OTP190PXZ8F2.0 size=578231 2023-01-24T10:52:24.168 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.167+0000 7fbe187898c0 1 < completed multipart upload key=HJGFMANXEK4E upload=OTP190PXZ8F2 tag=RRUCIOPGXI04 type=write size=1626807 2023-01-24T10:52:24.175 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.174+0000 7fbe187898c0 1 > prepared part key=_multipart_D92A7W1TH7X0.2~SETXS859KYN5.0 size=1048576 2023-01-24T10:52:24.177 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.176+0000 7fbe187898c0 1 > prepared part key=_multipart_D92A7W1TH7X0.2~SETXS859KYN5.1 size=1048576 2023-01-24T10:52:24.178 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.177+0000 7fbe187898c0 1 > prepared part key=_multipart_D92A7W1TH7X0.2~SETXS859KYN5.2 size=886534 2023-01-24T10:52:24.178 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.177+0000 7fbe187898c0 1 > prepared multipart upload key=D92A7W1TH7X0 upload=SETXS859KYN5 tag=313KJ92K1NY1 type=write size=4032262 2023-01-24T10:52:24.181 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.179+0000 7fbe187898c0 1 < suggested update operation key=YTEKYBINDI7I tag=XHTMSWTN19W9 type=del size=3930576 2023-01-24T10:52:24.182 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.181+0000 7fbe187898c0 1 > prepared operation key=A0DJSLNVH25F tag=81W9DE8NG6SG type=del size=1057473 2023-01-24T10:52:24.185 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.183+0000 7fbe187898c0 1 < suggested update operation key=VBIA0USNT4OA tag=S11MJVWVJBR1 type=del size=1704317 2023-01-24T10:52:24.186 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.184+0000 7fbe187898c0 1 > prepared operation key=TIE8E7PAI3NC tag=5DG86J8U7XMC type=write size=667146 2023-01-24T10:52:24.187 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.186+0000 7fbe187898c0 1 < canceled operation key=A0DJSLNVH25F tag=55Z4ZWWVDBSX type=del size=40660 2023-01-24T10:52:24.188 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.187+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=84X3L1OZL4XG type=del size=843252 2023-01-24T10:52:24.192 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.190+0000 7fbe187898c0 1 < completed part key=_multipart_Z4E99NU2FUTH.2~OQVVUCL89FM2.0 size=1048576 2023-01-24T10:52:24.194 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.192+0000 7fbe187898c0 1 < completed part key=_multipart_Z4E99NU2FUTH.2~OQVVUCL89FM2.1 size=594403 2023-01-24T10:52:24.199 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.198+0000 7fbe187898c0 1 < completed multipart upload key=Z4E99NU2FUTH upload=OQVVUCL89FM2 tag=C1W5Y9IZK4HB type=write size=2691555 2023-01-24T10:52:24.202 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.200+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=BWWCNQ2EV60O type=del size=1629418 2023-01-24T10:52:24.204 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.203+0000 7fbe187898c0 1 < completed delete operation key=5W2UE0AZTOM0 tag=3L3XQ8QNJNXA type=del 2023-01-24T10:52:24.207 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.206+0000 7fbe187898c0 1 > prepared operation key=5W2UE0AZTOM0 tag=9CBW0I9TX7F9 type=del size=246212 2023-01-24T10:52:24.209 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.207+0000 7fbe187898c0 1 < completed delete operation key=RV7FCL2UARTQ tag=LK3PMBD9T418 type=del 2023-01-24T10:52:24.210 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.209+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=QP094FKZO871 type=write size=58319 2023-01-24T10:52:24.211 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.210+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~Q7HAQ2I0LXE2.0 size=550334 2023-01-24T10:52:24.213 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.212+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=Q7HAQ2I0LXE2 tag=JYTWEUDJA436 type=write size=1598910 2023-01-24T10:52:24.214 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.213+0000 7fbe187898c0 1 > prepared operation key=9UV7IWZOV627 tag=KDB2BIOSPQNP type=del size=3196658 2023-01-24T10:52:24.216 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.215+0000 7fbe187898c0 1 < completed part key=_multipart_D92A7W1TH7X0.2~NOW8SVL01LER.0 size=241693 2023-01-24T10:52:24.218 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.217+0000 7fbe187898c0 1 < completed multipart upload key=D92A7W1TH7X0 upload=NOW8SVL01LER tag=GYTRGRRY2K99 type=write size=1290269 2023-01-24T10:52:24.219 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.218+0000 7fbe187898c0 1 > prepared operation key=FDZVWZ0V33EB tag=BTCL8F3NMGIX type=del size=1808839 2023-01-24T10:52:24.221 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.219+0000 7fbe187898c0 1 < canceled operation key=5W2UE0AZTOM0 tag=6LOW9OKY2PGI type=del size=842019 2023-01-24T10:52:24.223 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.222+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~LREN15A3DJKQ.0 size=1048576 2023-01-24T10:52:24.225 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.224+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~LREN15A3DJKQ.1 size=1048576 2023-01-24T10:52:24.226 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.225+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~LREN15A3DJKQ.2 size=683539 2023-01-24T10:52:24.226 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.225+0000 7fbe187898c0 1 > prepared multipart upload key=3GJVNT6TXTK6 upload=LREN15A3DJKQ tag=SNNF53IWY5AG type=write size=3829267 2023-01-24T10:52:24.227 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.226+0000 7fbe187898c0 1 < completed part key=_multipart_S7XZ2Q32GCMK.2~7YLM1NHMXHJ3.0 size=1048576 2023-01-24T10:52:24.228 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.227+0000 7fbe187898c0 1 < completed part key=_multipart_S7XZ2Q32GCMK.2~7YLM1NHMXHJ3.1 size=1048576 2023-01-24T10:52:24.229 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.228+0000 7fbe187898c0 1 < completed part key=_multipart_S7XZ2Q32GCMK.2~7YLM1NHMXHJ3.2 size=630495 2023-01-24T10:52:24.231 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.230+0000 7fbe187898c0 1 < completed multipart upload key=S7XZ2Q32GCMK upload=7YLM1NHMXHJ3 tag=Y6JUAOT6NNU5 type=write size=3776223 2023-01-24T10:52:24.233 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.231+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=4LRS204NB0PP type=del size=3842871 2023-01-24T10:52:24.234 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.233+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~A991IBGRKV72.0 size=1048576 2023-01-24T10:52:24.235 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.234+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~A991IBGRKV72.1 size=357774 2023-01-24T10:52:24.237 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.236+0000 7fbe187898c0 1 < completed multipart upload key=5W2UE0AZTOM0 upload=A991IBGRKV72 tag=9KODTIIINRXZ type=write size=2454926 2023-01-24T10:52:24.238 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.237+0000 7fbe187898c0 1 > prepared operation key=S7XZ2Q32GCMK tag=FCSCS6OF5PQ5 type=del size=3559815 2023-01-24T10:52:24.239 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.238+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~3L4JL16IDC8I.0 size=1048576 2023-01-24T10:52:24.241 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.239+0000 7fbe187898c0 1 < completed part key=_multipart_FDZVWZ0V33EB.2~3L4JL16IDC8I.1 size=372073 2023-01-24T10:52:24.243 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.242+0000 7fbe187898c0 1 < completed multipart upload key=FDZVWZ0V33EB upload=3L4JL16IDC8I tag=XG3TNCYSEGFZ type=write size=2469225 2023-01-24T10:52:24.246 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.244+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~4H95YJ0IMLVU.0 size=1048576 2023-01-24T10:52:24.247 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.246+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~4H95YJ0IMLVU.1 size=1048576 2023-01-24T10:52:24.248 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.247+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~4H95YJ0IMLVU.2 size=444212 2023-01-24T10:52:24.248 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.247+0000 7fbe187898c0 1 > prepared multipart upload key=YGC2UEGMODS8 upload=4H95YJ0IMLVU tag=RUNPTNTK1S4D type=write size=3589940 2023-01-24T10:52:24.250 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.248+0000 7fbe187898c0 1 < completed part key=_multipart_ZMEBN0OQ1IPJ.2~H37X886SQVB3.0 size=1048576 2023-01-24T10:52:24.251 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.249+0000 7fbe187898c0 1 < completed part key=_multipart_ZMEBN0OQ1IPJ.2~H37X886SQVB3.1 size=1048576 2023-01-24T10:52:24.252 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.250+0000 7fbe187898c0 1 < completed part key=_multipart_ZMEBN0OQ1IPJ.2~H37X886SQVB3.2 size=833483 2023-01-24T10:52:24.254 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.253+0000 7fbe187898c0 1 < completed multipart upload key=ZMEBN0OQ1IPJ upload=H37X886SQVB3 tag=O70MATOM1Y0O type=write size=3979211 2023-01-24T10:52:24.255 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.254+0000 7fbe187898c0 1 > prepared operation key=Z4E99NU2FUTH tag=MEZBR6NE942I type=write size=517328 2023-01-24T10:52:24.257 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.255+0000 7fbe187898c0 1 < completed delete operation key=Z8G7Y4RGVWR8 tag=SEJOT8N3TZIU type=del 2023-01-24T10:52:24.259 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.258+0000 7fbe187898c0 1 > prepared part key=_multipart_RV7FCL2UARTQ.2~1ZEL3KFZDQIN.0 size=490030 2023-01-24T10:52:24.259 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.258+0000 7fbe187898c0 1 > prepared multipart upload key=RV7FCL2UARTQ upload=1ZEL3KFZDQIN tag=DFQQW0THAOGS type=write size=1538606 2023-01-24T10:52:24.261 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.259+0000 7fbe187898c0 1 < completed delete operation key=63PDRJCRVEI1 tag=9R8CWWR818TP type=del 2023-01-24T10:52:24.263 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.262+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~D3VNJZ65SPLF.0 size=103398 2023-01-24T10:52:24.263 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.262+0000 7fbe187898c0 1 > prepared multipart upload key=TIE8E7PAI3NC upload=D3VNJZ65SPLF tag=NO8ZBIH5AX5M type=write size=1151974 2023-01-24T10:52:24.264 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.262+0000 7fbe187898c0 1 < canceled part key=_multipart_D92A7W1TH7X0.2~SETXS859KYN5.0 size=1048576 2023-01-24T10:52:24.266 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.264+0000 7fbe187898c0 1 < completed part key=_multipart_D92A7W1TH7X0.2~SETXS859KYN5.1 size=1048576 2023-01-24T10:52:24.269 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.267+0000 7fbe187898c0 1 < completed part key=_multipart_D92A7W1TH7X0.2~SETXS859KYN5.2 size=886534 2023-01-24T10:52:24.274 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.273+0000 7fbe187898c0 1 < canceled multipart upload key=D92A7W1TH7X0 upload=SETXS859KYN5 tag=313KJ92K1NY1 type=write size=4032262 2023-01-24T10:52:24.275 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.274+0000 7fbe187898c0 1 > prepared operation key=Z4E99NU2FUTH tag=VDG7OH8BOHFU type=write size=284279 2023-01-24T10:52:24.277 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.275+0000 7fbe187898c0 1 < completed delete operation key=A0DJSLNVH25F tag=81W9DE8NG6SG type=del 2023-01-24T10:52:24.278 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.277+0000 7fbe187898c0 1 > prepared operation key=XULE5KU5OITV tag=51E75UAEA1W8 type=del size=3085182 2023-01-24T10:52:24.279 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.278+0000 7fbe187898c0 1 < completed write operation key=TIE8E7PAI3NC tag=5DG86J8U7XMC type=write size=667146 2023-01-24T10:52:24.281 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.280+0000 7fbe187898c0 1 > prepared part key=_multipart_RBS906ROG13P.2~UFSKILJK9APT.0 size=1048576 2023-01-24T10:52:24.283 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.281+0000 7fbe187898c0 1 > prepared part key=_multipart_RBS906ROG13P.2~UFSKILJK9APT.1 size=1048576 2023-01-24T10:52:24.284 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.282+0000 7fbe187898c0 1 > prepared part key=_multipart_RBS906ROG13P.2~UFSKILJK9APT.2 size=928716 2023-01-24T10:52:24.284 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.282+0000 7fbe187898c0 1 > prepared multipart upload key=RBS906ROG13P upload=UFSKILJK9APT tag=QHO1TA8T5PT1 type=write size=4074444 2023-01-24T10:52:24.285 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.284+0000 7fbe187898c0 1 < completed delete operation key=THP93R0E799L tag=84X3L1OZL4XG type=del 2023-01-24T10:52:24.287 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.285+0000 7fbe187898c0 1 > prepared operation key=S7XZ2Q32GCMK tag=BDL2UXL5UD0N type=del size=104683 2023-01-24T10:52:24.288 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.287+0000 7fbe187898c0 1 < completed delete operation key=NU6FGGI1IUU7 tag=BWWCNQ2EV60O type=del 2023-01-24T10:52:24.289 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.288+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=EW6RJZGBE4XP type=del size=1777153 2023-01-24T10:52:24.290 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.289+0000 7fbe187898c0 1 < completed delete operation key=5W2UE0AZTOM0 tag=9CBW0I9TX7F9 type=del 2023-01-24T10:52:24.291 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.290+0000 7fbe187898c0 1 > prepared operation key=ZMEBN0OQ1IPJ tag=VUQZX47K11VS type=del size=3320592 2023-01-24T10:52:24.293 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.291+0000 7fbe187898c0 1 < completed write operation key=MQAFAWM6WGH1 tag=QP094FKZO871 type=write size=58319 2023-01-24T10:52:24.296 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.295+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~QC0P6ZZ393CI.0 size=1048576 2023-01-24T10:52:24.297 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.296+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~QC0P6ZZ393CI.1 size=1048576 2023-01-24T10:52:24.298 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.297+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~QC0P6ZZ393CI.2 size=143711 2023-01-24T10:52:24.298 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.297+0000 7fbe187898c0 1 > prepared multipart upload key=TIE8E7PAI3NC upload=QC0P6ZZ393CI tag=3TMQVU11AXQA type=write size=3289439 2023-01-24T10:52:24.299 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.298+0000 7fbe187898c0 1 < completed delete operation key=9UV7IWZOV627 tag=KDB2BIOSPQNP type=del 2023-01-24T10:52:24.300 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.299+0000 7fbe187898c0 1 > prepared operation key=3GJVNT6TXTK6 tag=72E14V804UAE type=write size=635005 2023-01-24T10:52:24.309 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.300+0000 7fbe187898c0 1 < completed delete operation key=FDZVWZ0V33EB tag=BTCL8F3NMGIX type=del 2023-01-24T10:52:24.309 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.302+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~PXJNQANYMBIM.0 size=108007 2023-01-24T10:52:24.309 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.302+0000 7fbe187898c0 1 > prepared multipart upload key=TIE8E7PAI3NC upload=PXJNQANYMBIM tag=YPBA92BNJ9RV type=write size=1156583 2023-01-24T10:52:24.310 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.304+0000 7fbe187898c0 1 < completed part key=_multipart_3GJVNT6TXTK6.2~LREN15A3DJKQ.0 size=1048576 2023-01-24T10:52:24.310 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.304+0000 7fbe187898c0 1 < canceled part key=_multipart_3GJVNT6TXTK6.2~LREN15A3DJKQ.1 size=1048576 2023-01-24T10:52:24.310 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.305+0000 7fbe187898c0 1 < completed part key=_multipart_3GJVNT6TXTK6.2~LREN15A3DJKQ.2 size=683539 2023-01-24T10:52:24.310 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.307+0000 7fbe187898c0 1 < canceled multipart upload key=3GJVNT6TXTK6 upload=LREN15A3DJKQ tag=SNNF53IWY5AG type=write size=3829267 2023-01-24T10:52:24.311 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.308+0000 7fbe187898c0 1 > prepared operation key=TIE8E7PAI3NC tag=LXKQRUQXV2HX type=del size=3818791 2023-01-24T10:52:24.311 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.309+0000 7fbe187898c0 1 < completed delete operation key=VBIA0USNT4OA tag=4LRS204NB0PP type=del 2023-01-24T10:52:24.312 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.311+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=HZZ7BB334JWR type=write size=403271 2023-01-24T10:52:24.313 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.312+0000 7fbe187898c0 1 < completed delete operation key=S7XZ2Q32GCMK tag=FCSCS6OF5PQ5 type=del 2023-01-24T10:52:24.315 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.313+0000 7fbe187898c0 1 > prepared operation key=RBS906ROG13P tag=BFGCKN7PQ5R0 type=del size=1632053 2023-01-24T10:52:24.315 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.313+0000 7fbe187898c0 1 < canceled part key=_multipart_YGC2UEGMODS8.2~4H95YJ0IMLVU.0 size=1048576 2023-01-24T10:52:24.316 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.315+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~4H95YJ0IMLVU.1 size=1048576 2023-01-24T10:52:24.317 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.316+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~4H95YJ0IMLVU.2 size=444212 2023-01-24T10:52:24.319 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.318+0000 7fbe187898c0 1 < canceled multipart upload key=YGC2UEGMODS8 upload=4H95YJ0IMLVU tag=RUNPTNTK1S4D type=write size=3589940 2023-01-24T10:52:24.322 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.320+0000 7fbe187898c0 1 > prepared part key=_multipart_ZMEBN0OQ1IPJ.2~UJ1UTEIS1X5G.0 size=942044 2023-01-24T10:52:24.322 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.320+0000 7fbe187898c0 1 > prepared multipart upload key=ZMEBN0OQ1IPJ upload=UJ1UTEIS1X5G tag=0KI8EYKE4U0M type=write size=1990620 2023-01-24T10:52:24.323 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.322+0000 7fbe187898c0 1 < canceled operation key=Z4E99NU2FUTH tag=MEZBR6NE942I type=write size=517328 2023-01-24T10:52:24.324 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.323+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=8VCZ6T3646VN type=del size=2515323 2023-01-24T10:52:24.325 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.324+0000 7fbe187898c0 1 < completed part key=_multipart_RV7FCL2UARTQ.2~1ZEL3KFZDQIN.0 size=490030 2023-01-24T10:52:24.327 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.326+0000 7fbe187898c0 1 < completed multipart upload key=RV7FCL2UARTQ upload=1ZEL3KFZDQIN tag=DFQQW0THAOGS type=write size=1538606 2023-01-24T10:52:24.329 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.327+0000 7fbe187898c0 1 > prepared operation key=0VR1WPQLHOV0 tag=9WNO1AQXJA4E type=write size=841150 2023-01-24T10:52:24.330 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.329+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~D3VNJZ65SPLF.0 size=103398 2023-01-24T10:52:24.332 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.331+0000 7fbe187898c0 1 < completed multipart upload key=TIE8E7PAI3NC upload=D3VNJZ65SPLF tag=NO8ZBIH5AX5M type=write size=1151974 2023-01-24T10:52:24.333 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.332+0000 7fbe187898c0 1 > prepared operation key=0VR1WPQLHOV0 tag=2E6LNZUS6U5D type=del size=2714339 2023-01-24T10:52:24.334 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.333+0000 7fbe187898c0 1 < completed write operation key=Z4E99NU2FUTH tag=VDG7OH8BOHFU type=write size=284279 2023-01-24T10:52:24.336 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.334+0000 7fbe187898c0 1 > prepared operation key=63PDRJCRVEI1 tag=VQIFHHBL017Y type=del size=2388989 2023-01-24T10:52:24.338 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.337+0000 7fbe187898c0 1 < suggested remove operation key=XULE5KU5OITV tag=51E75UAEA1W8 type=del 2023-01-24T10:52:24.341 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.339+0000 7fbe187898c0 1 > prepared part key=_multipart_XULE5KU5OITV.2~NJ6NXKFFY4FD.0 size=1048576 2023-01-24T10:52:24.341 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.340+0000 7fbe187898c0 1 > prepared part key=_multipart_XULE5KU5OITV.2~NJ6NXKFFY4FD.1 size=127932 2023-01-24T10:52:24.342 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.340+0000 7fbe187898c0 1 > prepared multipart upload key=XULE5KU5OITV upload=NJ6NXKFFY4FD tag=3DJI5Q0WGAK9 type=write size=2225084 2023-01-24T10:52:24.343 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.342+0000 7fbe187898c0 1 < completed part key=_multipart_RBS906ROG13P.2~UFSKILJK9APT.0 size=1048576 2023-01-24T10:52:24.346 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.345+0000 7fbe187898c0 1 < completed part key=_multipart_RBS906ROG13P.2~UFSKILJK9APT.1 size=1048576 2023-01-24T10:52:24.346 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.345+0000 7fbe187898c0 1 < canceled part key=_multipart_RBS906ROG13P.2~UFSKILJK9APT.2 size=928716 2023-01-24T10:52:24.350 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.348+0000 7fbe187898c0 1 < canceled multipart upload key=RBS906ROG13P upload=UFSKILJK9APT tag=QHO1TA8T5PT1 type=write size=4074444 2023-01-24T10:52:24.352 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.351+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=I5YKNJBCNRWQ type=del size=2946151 2023-01-24T10:52:24.355 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.354+0000 7fbe187898c0 1 < completed delete operation key=S7XZ2Q32GCMK tag=BDL2UXL5UD0N type=del 2023-01-24T10:52:24.358 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.357+0000 7fbe187898c0 1 > prepared operation key=HJGFMANXEK4E tag=XSUEOXSHQF90 type=del size=8212 2023-01-24T10:52:24.360 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.359+0000 7fbe187898c0 1 < completed delete operation key=WDQNB5TEJUC6 tag=EW6RJZGBE4XP type=del 2023-01-24T10:52:24.362 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.360+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=YMP78VZHZAM3 type=del size=2682992 2023-01-24T10:52:24.363 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.362+0000 7fbe187898c0 1 < canceled operation key=ZMEBN0OQ1IPJ tag=VUQZX47K11VS type=del size=3320592 2023-01-24T10:52:24.367 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.366+0000 7fbe187898c0 1 > prepared part key=_multipart_RBS906ROG13P.2~KPPWPC2JEBRP.0 size=1048576 2023-01-24T10:52:24.370 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.369+0000 7fbe187898c0 1 > prepared part key=_multipart_RBS906ROG13P.2~KPPWPC2JEBRP.1 size=665626 2023-01-24T10:52:24.370 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.369+0000 7fbe187898c0 1 > prepared multipart upload key=RBS906ROG13P upload=KPPWPC2JEBRP tag=0LM9YFTH6SMD type=write size=2762778 2023-01-24T10:52:24.373 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.372+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~QC0P6ZZ393CI.0 size=1048576 2023-01-24T10:52:24.373 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.372+0000 7fbe187898c0 1 < canceled part key=_multipart_TIE8E7PAI3NC.2~QC0P6ZZ393CI.1 size=1048576 2023-01-24T10:52:24.376 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.374+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~QC0P6ZZ393CI.2 size=143711 2023-01-24T10:52:24.381 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.380+0000 7fbe187898c0 1 < canceled multipart upload key=TIE8E7PAI3NC upload=QC0P6ZZ393CI tag=3TMQVU11AXQA type=write size=3289439 2023-01-24T10:52:24.383 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.381+0000 7fbe187898c0 1 > prepared operation key=ZQCEEZ3RIC8F tag=QPCCQWO366VL type=del size=461865 2023-01-24T10:52:24.388 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.387+0000 7fbe187898c0 1 < suggested update operation key=3GJVNT6TXTK6 tag=72E14V804UAE type=write size=635005 2023-01-24T10:52:24.391 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.390+0000 7fbe187898c0 1 > prepared operation key=SNBN0CZ6S9TA tag=48B94HVS1L2C type=del size=186168 2023-01-24T10:52:24.394 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.392+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~PXJNQANYMBIM.0 size=108007 2023-01-24T10:52:24.399 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.398+0000 7fbe187898c0 1 < completed multipart upload key=TIE8E7PAI3NC upload=PXJNQANYMBIM tag=YPBA92BNJ9RV type=write size=1156583 2023-01-24T10:52:24.402 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.401+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=P9C5IY23IILM type=write size=424905 2023-01-24T10:52:24.404 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.403+0000 7fbe187898c0 1 < completed delete operation key=TIE8E7PAI3NC tag=LXKQRUQXV2HX type=del 2023-01-24T10:52:24.410 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.409+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~5Z1CYRU9HS4O.0 size=1048576 2023-01-24T10:52:24.413 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.412+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~5Z1CYRU9HS4O.1 size=1013928 2023-01-24T10:52:24.413 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.412+0000 7fbe187898c0 1 > prepared multipart upload key=MQAFAWM6WGH1 upload=5Z1CYRU9HS4O tag=MFNS3KW54D4U type=write size=3111080 2023-01-24T10:52:24.416 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.415+0000 7fbe187898c0 1 < canceled operation key=YGC2UEGMODS8 tag=HZZ7BB334JWR type=write size=403271 2023-01-24T10:52:24.417 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.416+0000 7fbe187898c0 1 > prepared operation key=D92A7W1TH7X0 tag=0XEKBHHNKL49 type=del size=1089762 2023-01-24T10:52:24.420 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.419+0000 7fbe187898c0 1 < suggested remove operation key=RBS906ROG13P tag=BFGCKN7PQ5R0 type=del 2023-01-24T10:52:24.421 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.420+0000 7fbe187898c0 1 > prepared operation key=TIE8E7PAI3NC tag=HBQVM72NXIBB type=write size=295481 2023-01-24T10:52:24.423 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.421+0000 7fbe187898c0 1 < completed part key=_multipart_ZMEBN0OQ1IPJ.2~UJ1UTEIS1X5G.0 size=942044 2023-01-24T10:52:24.426 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.425+0000 7fbe187898c0 1 < completed multipart upload key=ZMEBN0OQ1IPJ upload=UJ1UTEIS1X5G tag=0KI8EYKE4U0M type=write size=1990620 2023-01-24T10:52:24.429 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.427+0000 7fbe187898c0 1 > prepared operation key=Z5DPD9MNHN2H tag=WROVYFSDVF4U type=del size=1515569 2023-01-24T10:52:24.434 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.433+0000 7fbe187898c0 1 < suggested update operation key=MQAFAWM6WGH1 tag=8VCZ6T3646VN type=del size=2515323 2023-01-24T10:52:24.437 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.436+0000 7fbe187898c0 1 > prepared operation key=SNBN0CZ6S9TA tag=EOU3M3AUG7QX type=del size=2954192 2023-01-24T10:52:24.440 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.439+0000 7fbe187898c0 1 < completed write operation key=0VR1WPQLHOV0 tag=9WNO1AQXJA4E type=write size=841150 2023-01-24T10:52:24.444 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.443+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~L1856W6J89TA.0 size=1048576 2023-01-24T10:52:24.446 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.445+0000 7fbe187898c0 1 > prepared part key=_multipart_YGC2UEGMODS8.2~L1856W6J89TA.1 size=855048 2023-01-24T10:52:24.447 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.445+0000 7fbe187898c0 1 > prepared multipart upload key=YGC2UEGMODS8 upload=L1856W6J89TA tag=CJ3UE4QYAJSN type=write size=2952200 2023-01-24T10:52:24.449 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.448+0000 7fbe187898c0 1 < completed delete operation key=0VR1WPQLHOV0 tag=2E6LNZUS6U5D type=del 2023-01-24T10:52:24.452 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.451+0000 7fbe187898c0 1 > prepared operation key=S7XZ2Q32GCMK tag=USNSB125LX98 type=del size=3109303 2023-01-24T10:52:24.455 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.453+0000 7fbe187898c0 1 < completed delete operation key=63PDRJCRVEI1 tag=VQIFHHBL017Y type=del 2023-01-24T10:52:24.457 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.456+0000 7fbe187898c0 1 > prepared operation key=C9J0MO49LDCS tag=VHD83QJTNWSQ type=write size=249234 2023-01-24T10:52:24.460 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.459+0000 7fbe187898c0 1 < completed part key=_multipart_XULE5KU5OITV.2~NJ6NXKFFY4FD.0 size=1048576 2023-01-24T10:52:24.463 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.461+0000 7fbe187898c0 1 < completed part key=_multipart_XULE5KU5OITV.2~NJ6NXKFFY4FD.1 size=127932 2023-01-24T10:52:24.468 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.466+0000 7fbe187898c0 1 < completed multipart upload key=XULE5KU5OITV upload=NJ6NXKFFY4FD tag=3DJI5Q0WGAK9 type=write size=2225084 2023-01-24T10:52:24.469 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.468+0000 7fbe187898c0 1 > prepared operation key=YTEKYBINDI7I tag=RIMX2H9QNB2O type=write size=146368 2023-01-24T10:52:24.471 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.470+0000 7fbe187898c0 1 < completed delete operation key=YGC2UEGMODS8 tag=I5YKNJBCNRWQ type=del 2023-01-24T10:52:24.473 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.472+0000 7fbe187898c0 1 > prepared part key=_multipart_XULE5KU5OITV.2~LLLVMPIJMKHF.0 size=1048576 2023-01-24T10:52:24.474 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.473+0000 7fbe187898c0 1 > prepared part key=_multipart_XULE5KU5OITV.2~LLLVMPIJMKHF.1 size=670730 2023-01-24T10:52:24.475 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.473+0000 7fbe187898c0 1 > prepared multipart upload key=XULE5KU5OITV upload=LLLVMPIJMKHF tag=NSMR612E0K07 type=write size=2767882 2023-01-24T10:52:24.475 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.474+0000 7fbe187898c0 1 < completed delete operation key=HJGFMANXEK4E tag=XSUEOXSHQF90 type=del 2023-01-24T10:52:24.477 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.475+0000 7fbe187898c0 1 > prepared operation key=C9J0MO49LDCS tag=3IC8QI35KZJ9 type=del size=2746842 2023-01-24T10:52:24.478 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.477+0000 7fbe187898c0 1 < completed delete operation key=VBIA0USNT4OA tag=YMP78VZHZAM3 type=del 2023-01-24T10:52:24.481 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.480+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~M92YI30I5YAF.0 size=1048576 2023-01-24T10:52:24.482 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.481+0000 7fbe187898c0 1 > prepared part key=_multipart_9UV7IWZOV627.2~M92YI30I5YAF.1 size=438572 2023-01-24T10:52:24.482 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.481+0000 7fbe187898c0 1 > prepared multipart upload key=9UV7IWZOV627 upload=M92YI30I5YAF tag=QPRK7CFI9L8H type=write size=2535724 2023-01-24T10:52:24.482 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.481+0000 7fbe187898c0 1 < canceled part key=_multipart_RBS906ROG13P.2~KPPWPC2JEBRP.0 size=1048576 2023-01-24T10:52:24.483 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.482+0000 7fbe187898c0 1 < completed part key=_multipart_RBS906ROG13P.2~KPPWPC2JEBRP.1 size=665626 2023-01-24T10:52:24.487 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.484+0000 7fbe187898c0 1 < canceled multipart upload key=RBS906ROG13P upload=KPPWPC2JEBRP tag=0LM9YFTH6SMD type=write size=2762778 2023-01-24T10:52:24.488 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.487+0000 7fbe187898c0 1 > prepared part key=_multipart_NU6FGGI1IUU7.2~EF57CURDRFNL.0 size=891959 2023-01-24T10:52:24.488 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.487+0000 7fbe187898c0 1 > prepared multipart upload key=NU6FGGI1IUU7 upload=EF57CURDRFNL tag=LXC8PBC18FK3 type=write size=1940535 2023-01-24T10:52:24.489 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.488+0000 7fbe187898c0 1 < completed delete operation key=ZQCEEZ3RIC8F tag=QPCCQWO366VL type=del 2023-01-24T10:52:24.490 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.489+0000 7fbe187898c0 1 > prepared operation key=Z5DPD9MNHN2H tag=AVEW63BOBRPF type=del size=3558858 2023-01-24T10:52:24.492 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.491+0000 7fbe187898c0 1 < completed delete operation key=SNBN0CZ6S9TA tag=48B94HVS1L2C type=del 2023-01-24T10:52:24.493 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.492+0000 7fbe187898c0 1 > prepared operation key=5W2UE0AZTOM0 tag=989QO5CJT43S type=del size=2361195 2023-01-24T10:52:24.495 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.493+0000 7fbe187898c0 1 < canceled operation key=WDQNB5TEJUC6 tag=P9C5IY23IILM type=write size=424905 2023-01-24T10:52:24.497 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.495+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~BZ4T47HHBGXC.0 size=1048576 2023-01-24T10:52:24.498 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.496+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~BZ4T47HHBGXC.1 size=1048576 2023-01-24T10:52:24.499 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.498+0000 7fbe187898c0 1 > prepared part key=_multipart_3GJVNT6TXTK6.2~BZ4T47HHBGXC.2 size=419458 2023-01-24T10:52:24.500 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.498+0000 7fbe187898c0 1 > prepared multipart upload key=3GJVNT6TXTK6 upload=BZ4T47HHBGXC tag=55X65FGBJZ96 type=write size=3565186 2023-01-24T10:52:24.500 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.499+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~5Z1CYRU9HS4O.0 size=1048576 2023-01-24T10:52:24.502 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.501+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~5Z1CYRU9HS4O.1 size=1013928 2023-01-24T10:52:24.507 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.506+0000 7fbe187898c0 1 < completed multipart upload key=MQAFAWM6WGH1 upload=5Z1CYRU9HS4O tag=MFNS3KW54D4U type=write size=3111080 2023-01-24T10:52:24.513 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.512+0000 7fbe187898c0 1 > prepared part key=_multipart_5W2UE0AZTOM0.2~ICYEK5ORQBK4.0 size=307286 2023-01-24T10:52:24.513 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.512+0000 7fbe187898c0 1 > prepared multipart upload key=5W2UE0AZTOM0 upload=ICYEK5ORQBK4 tag=FWCCKMCFPU7A type=write size=1355862 2023-01-24T10:52:24.516 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.514+0000 7fbe187898c0 1 < completed delete operation key=D92A7W1TH7X0 tag=0XEKBHHNKL49 type=del 2023-01-24T10:52:24.518 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.517+0000 7fbe187898c0 1 > prepared operation key=S7XZ2Q32GCMK tag=Q4443J21DSIG type=del size=3347280 2023-01-24T10:52:24.521 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.520+0000 7fbe187898c0 1 < completed write operation key=TIE8E7PAI3NC tag=HBQVM72NXIBB type=write size=295481 2023-01-24T10:52:24.524 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.523+0000 7fbe187898c0 1 > prepared operation key=3GJVNT6TXTK6 tag=L1BRG4VZTT3S type=write size=134203 2023-01-24T10:52:24.527 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.525+0000 7fbe187898c0 1 < completed delete operation key=Z5DPD9MNHN2H tag=WROVYFSDVF4U type=del 2023-01-24T10:52:24.529 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.528+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=4C5KNOI7NP6G type=del size=323216 2023-01-24T10:52:24.532 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.531+0000 7fbe187898c0 1 < completed delete operation key=SNBN0CZ6S9TA tag=EOU3M3AUG7QX type=del 2023-01-24T10:52:24.533 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.532+0000 7fbe187898c0 1 > prepared operation key=0VR1WPQLHOV0 tag=U8J9G69CQKUM type=del size=2288113 2023-01-24T10:52:24.535 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.533+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~L1856W6J89TA.0 size=1048576 2023-01-24T10:52:24.536 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.535+0000 7fbe187898c0 1 < completed part key=_multipart_YGC2UEGMODS8.2~L1856W6J89TA.1 size=855048 2023-01-24T10:52:24.542 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.541+0000 7fbe187898c0 1 < completed multipart upload key=YGC2UEGMODS8 upload=L1856W6J89TA tag=CJ3UE4QYAJSN type=write size=2952200 2023-01-24T10:52:24.545 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.543+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=50BTTLHAATQ4 type=del size=3451057 2023-01-24T10:52:24.548 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.546+0000 7fbe187898c0 1 < completed delete operation key=S7XZ2Q32GCMK tag=USNSB125LX98 type=del 2023-01-24T10:52:24.550 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.549+0000 7fbe187898c0 1 > prepared operation key=XULE5KU5OITV tag=6WB5ULFZWO0M type=del size=2379958 2023-01-24T10:52:24.556 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.554+0000 7fbe187898c0 1 < suggested update operation key=C9J0MO49LDCS tag=VHD83QJTNWSQ type=write size=249234 2023-01-24T10:52:24.558 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.557+0000 7fbe187898c0 1 > prepared operation key=YGC2UEGMODS8 tag=YTRLICE4PKKO type=del size=3215807 2023-01-24T10:52:24.561 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.560+0000 7fbe187898c0 1 < completed write operation key=YTEKYBINDI7I tag=RIMX2H9QNB2O type=write size=146368 2023-01-24T10:52:24.563 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.562+0000 7fbe187898c0 1 > prepared operation key=ZMEBN0OQ1IPJ tag=R12M690QHMQG type=del size=704211 2023-01-24T10:52:24.566 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.565+0000 7fbe187898c0 1 < completed part key=_multipart_XULE5KU5OITV.2~LLLVMPIJMKHF.0 size=1048576 2023-01-24T10:52:24.566 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.565+0000 7fbe187898c0 1 < canceled part key=_multipart_XULE5KU5OITV.2~LLLVMPIJMKHF.1 size=670730 2023-01-24T10:52:24.575 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.570+0000 7fbe187898c0 1 < canceled multipart upload key=XULE5KU5OITV upload=LLLVMPIJMKHF tag=NSMR612E0K07 type=write size=2767882 2023-01-24T10:52:24.575 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.573+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=IQO0QP1CUHX1 type=del size=1931472 2023-01-24T10:52:24.577 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.575+0000 7fbe187898c0 -1 < failed to complete operation key=C9J0MO49LDCS tag=3IC8QI35KZJ9 type=del size=2746842: (22) Invalid argument 2023-01-24T10:52:24.582 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.581+0000 7fbe187898c0 1 > prepared part key=_multipart_S7XZ2Q32GCMK.2~2ZJ2C07SUH01.0 size=698690 2023-01-24T10:52:24.582 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.581+0000 7fbe187898c0 1 > prepared multipart upload key=S7XZ2Q32GCMK upload=2ZJ2C07SUH01 tag=TVN5N6MD2OP6 type=write size=1747266 2023-01-24T10:52:24.585 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.583+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~M92YI30I5YAF.0 size=1048576 2023-01-24T10:52:24.587 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.586+0000 7fbe187898c0 1 < completed part key=_multipart_9UV7IWZOV627.2~M92YI30I5YAF.1 size=438572 2023-01-24T10:52:24.590 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.588+0000 7fbe187898c0 1 < completed multipart upload key=9UV7IWZOV627 upload=M92YI30I5YAF tag=QPRK7CFI9L8H type=write size=2535724 2023-01-24T10:52:24.592 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.591+0000 7fbe187898c0 1 > prepared part key=_multipart_XULE5KU5OITV.2~A2NL3LGK26U6.0 size=131953 2023-01-24T10:52:24.592 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.591+0000 7fbe187898c0 1 > prepared multipart upload key=XULE5KU5OITV upload=A2NL3LGK26U6 tag=9U6JSXKS83HF type=write size=1180529 2023-01-24T10:52:24.595 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.594+0000 7fbe187898c0 1 < completed part key=_multipart_NU6FGGI1IUU7.2~EF57CURDRFNL.0 size=891959 2023-01-24T10:52:24.600 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.599+0000 7fbe187898c0 1 < completed multipart upload key=NU6FGGI1IUU7 upload=EF57CURDRFNL tag=LXC8PBC18FK3 type=write size=1940535 2023-01-24T10:52:24.606 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.604+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~3PPEV47WO8U2.0 size=1048576 2023-01-24T10:52:24.608 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.607+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~3PPEV47WO8U2.1 size=1048576 2023-01-24T10:52:24.611 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.610+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~3PPEV47WO8U2.2 size=869509 2023-01-24T10:52:24.611 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.610+0000 7fbe187898c0 1 > prepared multipart upload key=TIE8E7PAI3NC upload=3PPEV47WO8U2 tag=E0G98E3BF9WP type=write size=4015237 2023-01-24T10:52:24.612 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.611+0000 7fbe187898c0 1 < canceled operation key=Z5DPD9MNHN2H tag=AVEW63BOBRPF type=del size=3558858 2023-01-24T10:52:24.614 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.613+0000 7fbe187898c0 1 > prepared operation key=GNT28K0GNFMJ tag=1CK5TITMQEBD type=del size=2647744 2023-01-24T10:52:24.615 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.614+0000 7fbe187898c0 1 < canceled operation key=5W2UE0AZTOM0 tag=989QO5CJT43S type=del size=2361195 2023-01-24T10:52:24.616 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.615+0000 7fbe187898c0 1 > prepared operation key=MQAFAWM6WGH1 tag=SXAQ0YZ0IJGX type=del size=3604175 2023-01-24T10:52:24.617 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.616+0000 7fbe187898c0 1 < canceled part key=_multipart_3GJVNT6TXTK6.2~BZ4T47HHBGXC.0 size=1048576 2023-01-24T10:52:24.620 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.617+0000 7fbe187898c0 1 < completed part key=_multipart_3GJVNT6TXTK6.2~BZ4T47HHBGXC.1 size=1048576 2023-01-24T10:52:24.620 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.618+0000 7fbe187898c0 1 < completed part key=_multipart_3GJVNT6TXTK6.2~BZ4T47HHBGXC.2 size=419458 2023-01-24T10:52:24.625 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.624+0000 7fbe187898c0 1 < canceled multipart upload key=3GJVNT6TXTK6 upload=BZ4T47HHBGXC tag=55X65FGBJZ96 type=write size=3565186 2023-01-24T10:52:24.627 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.626+0000 7fbe187898c0 1 > prepared part key=_multipart_GNT28K0GNFMJ.2~ZYLCZXXFO8IM.0 size=1048576 2023-01-24T10:52:24.629 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.627+0000 7fbe187898c0 1 > prepared part key=_multipart_GNT28K0GNFMJ.2~ZYLCZXXFO8IM.1 size=1048576 2023-01-24T10:52:24.630 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.629+0000 7fbe187898c0 1 > prepared part key=_multipart_GNT28K0GNFMJ.2~ZYLCZXXFO8IM.2 size=159337 2023-01-24T10:52:24.630 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.629+0000 7fbe187898c0 1 > prepared multipart upload key=GNT28K0GNFMJ upload=ZYLCZXXFO8IM tag=W8604ZEQD1ZH type=write size=3305065 2023-01-24T10:52:24.631 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.630+0000 7fbe187898c0 1 < completed part key=_multipart_5W2UE0AZTOM0.2~ICYEK5ORQBK4.0 size=307286 2023-01-24T10:52:24.633 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.632+0000 7fbe187898c0 1 < completed multipart upload key=5W2UE0AZTOM0 upload=ICYEK5ORQBK4 tag=FWCCKMCFPU7A type=write size=1355862 2023-01-24T10:52:24.635 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.634+0000 7fbe187898c0 1 > prepared operation key=0VR1WPQLHOV0 tag=9C5BD0F6ATHB type=del size=3745639 2023-01-24T10:52:24.637 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.636+0000 7fbe187898c0 1 < completed delete operation key=S7XZ2Q32GCMK tag=Q4443J21DSIG type=del 2023-01-24T10:52:24.638 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.637+0000 7fbe187898c0 1 > prepared operation key=ZMEBN0OQ1IPJ tag=47GFFJILFQ2F type=del size=1756757 2023-01-24T10:52:24.640 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.638+0000 7fbe187898c0 1 < completed write operation key=3GJVNT6TXTK6 tag=L1BRG4VZTT3S type=write size=134203 2023-01-24T10:52:24.641 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.640+0000 7fbe187898c0 1 > prepared operation key=0VR1WPQLHOV0 tag=R655RY2XMHNL type=del size=194754 2023-01-24T10:52:24.643 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.641+0000 7fbe187898c0 1 < completed delete operation key=MQAFAWM6WGH1 tag=4C5KNOI7NP6G type=del 2023-01-24T10:52:24.644 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.642+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=TSP97V45AWGT type=del size=3276882 2023-01-24T10:52:24.648 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.647+0000 7fbe187898c0 1 < suggested update operation key=0VR1WPQLHOV0 tag=U8J9G69CQKUM type=del size=2288113 2023-01-24T10:52:24.651 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.650+0000 7fbe187898c0 1 > prepared operation key=K1AKVEEHDJEG tag=778O0Z3XQ2ZB type=del size=1870518 2023-01-24T10:52:24.653 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.652+0000 7fbe187898c0 1 < completed delete operation key=MQAFAWM6WGH1 tag=50BTTLHAATQ4 type=del 2023-01-24T10:52:24.657 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.656+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~ZAEJNAR60SEZ.0 size=1048576 2023-01-24T10:52:24.660 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.659+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~ZAEJNAR60SEZ.1 size=1048576 2023-01-24T10:52:24.662 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.660+0000 7fbe187898c0 1 > prepared part key=_multipart_VBIA0USNT4OA.2~ZAEJNAR60SEZ.2 size=1035375 2023-01-24T10:52:24.662 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.660+0000 7fbe187898c0 1 > prepared multipart upload key=VBIA0USNT4OA upload=ZAEJNAR60SEZ tag=DT7R50364T42 type=write size=4181103 2023-01-24T10:52:24.664 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.663+0000 7fbe187898c0 1 < completed delete operation key=XULE5KU5OITV tag=6WB5ULFZWO0M type=del 2023-01-24T10:52:24.670 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.668+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~CM6P0UG2MLX1.0 size=1048576 2023-01-24T10:52:24.671 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.670+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~CM6P0UG2MLX1.1 size=1048576 2023-01-24T10:52:24.672 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.671+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~CM6P0UG2MLX1.2 size=644214 2023-01-24T10:52:24.673 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.671+0000 7fbe187898c0 1 > prepared multipart upload key=Z5DPD9MNHN2H upload=CM6P0UG2MLX1 tag=EMHGR39VLPUP type=write size=3789942 2023-01-24T10:52:24.674 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.672+0000 7fbe187898c0 1 < canceled operation key=YGC2UEGMODS8 tag=YTRLICE4PKKO type=del size=3215807 2023-01-24T10:52:24.676 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.675+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~39ZSX1UQTZF4.0 size=1048576 2023-01-24T10:52:24.677 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.676+0000 7fbe187898c0 1 > prepared part key=_multipart_TIE8E7PAI3NC.2~39ZSX1UQTZF4.1 size=315236 2023-01-24T10:52:24.677 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.676+0000 7fbe187898c0 1 > prepared multipart upload key=TIE8E7PAI3NC upload=39ZSX1UQTZF4 tag=H6TNUL28V6K4 type=write size=2412388 2023-01-24T10:52:24.679 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.677+0000 7fbe187898c0 1 < completed delete operation key=ZMEBN0OQ1IPJ tag=R12M690QHMQG type=del 2023-01-24T10:52:24.680 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.679+0000 7fbe187898c0 1 > prepared operation key=63PDRJCRVEI1 tag=LQ4E6U7XF6ID type=del size=2107461 2023-01-24T10:52:24.681 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.680+0000 7fbe187898c0 1 < completed delete operation key=VBIA0USNT4OA tag=IQO0QP1CUHX1 type=del 2023-01-24T10:52:24.683 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.682+0000 7fbe187898c0 1 > prepared operation key=GNT28K0GNFMJ tag=BON2T7CX9BQL type=del size=725647 2023-01-24T10:52:24.683 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.682+0000 7fbe187898c0 1 < canceled part key=_multipart_S7XZ2Q32GCMK.2~2ZJ2C07SUH01.0 size=698690 2023-01-24T10:52:24.687 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.686+0000 7fbe187898c0 1 < canceled multipart upload key=S7XZ2Q32GCMK upload=2ZJ2C07SUH01 tag=TVN5N6MD2OP6 type=write size=1747266 2023-01-24T10:52:24.691 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.690+0000 7fbe187898c0 1 > prepared part key=_multipart_SNBN0CZ6S9TA.2~GC1EDHQVFW7W.0 size=1048576 2023-01-24T10:52:24.692 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.691+0000 7fbe187898c0 1 > prepared part key=_multipart_SNBN0CZ6S9TA.2~GC1EDHQVFW7W.1 size=1048576 2023-01-24T10:52:24.693 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.692+0000 7fbe187898c0 1 > prepared part key=_multipart_SNBN0CZ6S9TA.2~GC1EDHQVFW7W.2 size=280166 2023-01-24T10:52:24.694 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.692+0000 7fbe187898c0 1 > prepared multipart upload key=SNBN0CZ6S9TA upload=GC1EDHQVFW7W tag=EVI31PP134XY type=write size=3425894 2023-01-24T10:52:24.694 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.692+0000 7fbe187898c0 1 < canceled part key=_multipart_XULE5KU5OITV.2~A2NL3LGK26U6.0 size=131953 2023-01-24T10:52:24.696 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.694+0000 7fbe187898c0 1 < canceled multipart upload key=XULE5KU5OITV upload=A2NL3LGK26U6 tag=9U6JSXKS83HF type=write size=1180529 2023-01-24T10:52:24.698 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.697+0000 7fbe187898c0 1 > prepared part key=_multipart_S7XZ2Q32GCMK.2~OIUS5SVRW4SB.0 size=922528 2023-01-24T10:52:24.699 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.697+0000 7fbe187898c0 1 > prepared multipart upload key=S7XZ2Q32GCMK upload=OIUS5SVRW4SB tag=H4BB5ZMOKQCH type=write size=1971104 2023-01-24T10:52:24.700 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.698+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~3PPEV47WO8U2.0 size=1048576 2023-01-24T10:52:24.700 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.698+0000 7fbe187898c0 1 < canceled part key=_multipart_TIE8E7PAI3NC.2~3PPEV47WO8U2.1 size=1048576 2023-01-24T10:52:24.701 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.699+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~3PPEV47WO8U2.2 size=869509 2023-01-24T10:52:24.703 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.702+0000 7fbe187898c0 1 < canceled multipart upload key=TIE8E7PAI3NC upload=3PPEV47WO8U2 tag=E0G98E3BF9WP type=write size=4015237 2023-01-24T10:52:24.706 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.704+0000 7fbe187898c0 1 > prepared part key=_multipart_C9J0MO49LDCS.2~TYRGA1ORS15O.0 size=748846 2023-01-24T10:52:24.706 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.704+0000 7fbe187898c0 1 > prepared multipart upload key=C9J0MO49LDCS upload=TYRGA1ORS15O tag=I6FCN1N36IA0 type=write size=1797422 2023-01-24T10:52:24.707 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.706+0000 7fbe187898c0 1 < completed delete operation key=GNT28K0GNFMJ tag=1CK5TITMQEBD type=del 2023-01-24T10:52:24.713 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.708+0000 7fbe187898c0 1 > prepared operation key=Z4E99NU2FUTH tag=6DBQ131ABV7W type=del size=2072083 2023-01-24T10:52:24.713 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.710+0000 7fbe187898c0 1 < completed delete operation key=MQAFAWM6WGH1 tag=SXAQ0YZ0IJGX type=del 2023-01-24T10:52:24.718 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.713+0000 7fbe187898c0 1 > prepared part key=_multipart_MQAFAWM6WGH1.2~7GSIEZLHAPVH.0 size=358937 2023-01-24T10:52:24.718 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.713+0000 7fbe187898c0 1 > prepared multipart upload key=MQAFAWM6WGH1 upload=7GSIEZLHAPVH tag=ATJZKDZ8TP18 type=write size=1407513 2023-01-24T10:52:24.718 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.714+0000 7fbe187898c0 1 < canceled part key=_multipart_GNT28K0GNFMJ.2~ZYLCZXXFO8IM.0 size=1048576 2023-01-24T10:52:24.719 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.715+0000 7fbe187898c0 1 < completed part key=_multipart_GNT28K0GNFMJ.2~ZYLCZXXFO8IM.1 size=1048576 2023-01-24T10:52:24.719 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.718+0000 7fbe187898c0 1 < completed part key=_multipart_GNT28K0GNFMJ.2~ZYLCZXXFO8IM.2 size=159337 2023-01-24T10:52:24.724 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.722+0000 7fbe187898c0 1 < canceled multipart upload key=GNT28K0GNFMJ upload=ZYLCZXXFO8IM tag=W8604ZEQD1ZH type=write size=3305065 2023-01-24T10:52:24.726 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.724+0000 7fbe187898c0 1 > prepared operation key=5W2UE0AZTOM0 tag=5IGKGJAB86PK type=del size=3811079 2023-01-24T10:52:24.728 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.727+0000 7fbe187898c0 -1 < failed to complete operation key=0VR1WPQLHOV0 tag=9C5BD0F6ATHB type=del size=3745639: (22) Invalid argument 2023-01-24T10:52:24.729 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.728+0000 7fbe187898c0 1 > prepared operation key=TIE8E7PAI3NC tag=89QS1Z0FLOP8 type=write size=292386 2023-01-24T10:52:24.732 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.731+0000 7fbe187898c0 1 < completed delete operation key=ZMEBN0OQ1IPJ tag=47GFFJILFQ2F type=del 2023-01-24T10:52:24.735 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.733+0000 7fbe187898c0 1 > prepared operation key=1ZN0Y6AS7ISA tag=2DN7TGXUVI4T type=del size=2199889 2023-01-24T10:52:24.736 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.735+0000 7fbe187898c0 -1 < failed to complete operation key=0VR1WPQLHOV0 tag=R655RY2XMHNL type=del size=194754: (22) Invalid argument 2023-01-24T10:52:24.738 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.737+0000 7fbe187898c0 1 > prepared operation key=Z8G7Y4RGVWR8 tag=ZXD1QXLYEDYM type=del size=2508175 2023-01-24T10:52:24.740 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.738+0000 7fbe187898c0 1 < completed delete operation key=THP93R0E799L tag=TSP97V45AWGT type=del 2023-01-24T10:52:24.741 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.740+0000 7fbe187898c0 1 > prepared operation key=XULE5KU5OITV tag=M45V8O4D9TF5 type=del size=2391614 2023-01-24T10:52:24.743 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.741+0000 7fbe187898c0 1 < completed delete operation key=K1AKVEEHDJEG tag=778O0Z3XQ2ZB type=del 2023-01-24T10:52:24.745 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.744+0000 7fbe187898c0 1 > prepared part key=_multipart_A0DJSLNVH25F.2~GDGFTG34PSD6.0 size=1048576 2023-01-24T10:52:24.747 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.745+0000 7fbe187898c0 1 > prepared part key=_multipart_A0DJSLNVH25F.2~GDGFTG34PSD6.1 size=97575 2023-01-24T10:52:24.747 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.745+0000 7fbe187898c0 1 > prepared multipart upload key=A0DJSLNVH25F upload=GDGFTG34PSD6 tag=P2LN17VN5LC7 type=write size=2194727 2023-01-24T10:52:24.749 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.747+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~ZAEJNAR60SEZ.0 size=1048576 2023-01-24T10:52:24.749 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.748+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~ZAEJNAR60SEZ.1 size=1048576 2023-01-24T10:52:24.752 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.751+0000 7fbe187898c0 1 < completed part key=_multipart_VBIA0USNT4OA.2~ZAEJNAR60SEZ.2 size=1035375 2023-01-24T10:52:24.758 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.757+0000 7fbe187898c0 1 < completed multipart upload key=VBIA0USNT4OA upload=ZAEJNAR60SEZ tag=DT7R50364T42 type=write size=4181103 2023-01-24T10:52:24.760 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.759+0000 7fbe187898c0 1 > prepared operation key=K1AKVEEHDJEG tag=43P0D5IN1QV6 type=del size=1890649 2023-01-24T10:52:24.764 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.762+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~CM6P0UG2MLX1.0 size=1048576 2023-01-24T10:52:24.764 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.762+0000 7fbe187898c0 1 < canceled part key=_multipart_Z5DPD9MNHN2H.2~CM6P0UG2MLX1.1 size=1048576 2023-01-24T10:52:24.766 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.765+0000 7fbe187898c0 1 < completed part key=_multipart_Z5DPD9MNHN2H.2~CM6P0UG2MLX1.2 size=644214 2023-01-24T10:52:24.771 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.769+0000 7fbe187898c0 1 < canceled multipart upload key=Z5DPD9MNHN2H upload=CM6P0UG2MLX1 tag=EMHGR39VLPUP type=write size=3789942 2023-01-24T10:52:24.772 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.771+0000 7fbe187898c0 1 > prepared operation key=WDQNB5TEJUC6 tag=SA2C08HKT61G type=del size=2984953 2023-01-24T10:52:24.774 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.772+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~39ZSX1UQTZF4.0 size=1048576 2023-01-24T10:52:24.775 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.774+0000 7fbe187898c0 1 < completed part key=_multipart_TIE8E7PAI3NC.2~39ZSX1UQTZF4.1 size=315236 2023-01-24T10:52:24.778 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.776+0000 7fbe187898c0 1 < completed multipart upload key=TIE8E7PAI3NC upload=39ZSX1UQTZF4 tag=H6TNUL28V6K4 type=write size=2412388 2023-01-24T10:52:24.780 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.779+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~JN3T184LXM2I.0 size=1048576 2023-01-24T10:52:24.782 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.780+0000 7fbe187898c0 1 > prepared part key=_multipart_Z5DPD9MNHN2H.2~JN3T184LXM2I.1 size=628207 2023-01-24T10:52:24.782 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.780+0000 7fbe187898c0 1 > prepared multipart upload key=Z5DPD9MNHN2H upload=JN3T184LXM2I tag=TH5K87HUHD4G type=write size=2725359 2023-01-24T10:52:24.783 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.782+0000 7fbe187898c0 1 < completed delete operation key=63PDRJCRVEI1 tag=LQ4E6U7XF6ID type=del 2023-01-24T10:52:24.785 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.784+0000 7fbe187898c0 1 > prepared operation key=C9J0MO49LDCS tag=GBHAI9XGAJ5R type=write size=650301 2023-01-24T10:52:24.786 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.785+0000 7fbe187898c0 1 < completed delete operation key=GNT28K0GNFMJ tag=BON2T7CX9BQL type=del 2023-01-24T10:52:24.788 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.786+0000 7fbe187898c0 1 > prepared operation key=VBIA0USNT4OA tag=3TS3F7OD2LHC type=del size=3892998 2023-01-24T10:52:24.790 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.788+0000 7fbe187898c0 1 < completed part key=_multipart_SNBN0CZ6S9TA.2~GC1EDHQVFW7W.0 size=1048576 2023-01-24T10:52:24.791 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.789+0000 7fbe187898c0 1 < completed part key=_multipart_SNBN0CZ6S9TA.2~GC1EDHQVFW7W.1 size=1048576 2023-01-24T10:52:24.793 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.791+0000 7fbe187898c0 1 < completed part key=_multipart_SNBN0CZ6S9TA.2~GC1EDHQVFW7W.2 size=280166 2023-01-24T10:52:24.795 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.793+0000 7fbe187898c0 1 < completed multipart upload key=SNBN0CZ6S9TA upload=GC1EDHQVFW7W tag=EVI31PP134XY type=write size=3425894 2023-01-24T10:52:24.796 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.795+0000 7fbe187898c0 1 > prepared operation key=Z5DPD9MNHN2H tag=4DIG0GOG48JB type=del size=3427007 2023-01-24T10:52:24.797 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.796+0000 7fbe187898c0 1 < completed part key=_multipart_S7XZ2Q32GCMK.2~OIUS5SVRW4SB.0 size=922528 2023-01-24T10:52:24.800 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.799+0000 7fbe187898c0 1 < completed multipart upload key=S7XZ2Q32GCMK upload=OIUS5SVRW4SB tag=H4BB5ZMOKQCH type=write size=1971104 2023-01-24T10:52:24.801 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.800+0000 7fbe187898c0 1 > prepared operation key=D0CGZ9OQK1AM tag=BEFXD8EU0FHS type=write size=739391 2023-01-24T10:52:24.802 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.800+0000 7fbe187898c0 1 < canceled part key=_multipart_C9J0MO49LDCS.2~TYRGA1ORS15O.0 size=748846 2023-01-24T10:52:24.804 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.803+0000 7fbe187898c0 1 < canceled multipart upload key=C9J0MO49LDCS upload=TYRGA1ORS15O tag=I6FCN1N36IA0 type=write size=1797422 2023-01-24T10:52:24.807 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.805+0000 7fbe187898c0 1 > prepared part key=_multipart_0VR1WPQLHOV0.2~AC71NL1XJYIX.0 size=1048576 2023-01-24T10:52:24.808 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.807+0000 7fbe187898c0 1 > prepared part key=_multipart_0VR1WPQLHOV0.2~AC71NL1XJYIX.1 size=1037321 2023-01-24T10:52:24.808 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.807+0000 7fbe187898c0 1 > prepared multipart upload key=0VR1WPQLHOV0 upload=AC71NL1XJYIX tag=90AJC5HPL6EA type=write size=3134473 2023-01-24T10:52:24.809 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.808+0000 7fbe187898c0 1 < completed delete operation key=Z4E99NU2FUTH tag=6DBQ131ABV7W type=del 2023-01-24T10:52:24.812 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.811+0000 7fbe187898c0 1 > prepared operation key=NU6FGGI1IUU7 tag=I0Y22LN87N6P type=write size=12117 2023-01-24T10:52:24.814 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.813+0000 7fbe187898c0 1 < completed part key=_multipart_MQAFAWM6WGH1.2~7GSIEZLHAPVH.0 size=358937 2023-01-24T10:52:24.818 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.817+0000 7fbe187898c0 1 < completed multipart upload key=MQAFAWM6WGH1 upload=7GSIEZLHAPVH tag=ATJZKDZ8TP18 type=write size=1407513 2023-01-24T10:52:24.819 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.818+0000 7fbe187898c0 1 > prepared operation key=THP93R0E799L tag=ABL7CSV101M8 type=del size=1573857 2023-01-24T10:52:24.822 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.821+0000 7fbe187898c0 1 < suggested remove operation key=5W2UE0AZTOM0 tag=5IGKGJAB86PK type=del 2023-01-24T10:52:24.824 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.822+0000 7fbe187898c0 1 > prepared operation key=XULE5KU5OITV tag=1DTY46YPZVH5 type=del size=2255744 2023-01-24T10:52:24.826 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.824+0000 7fbe187898c0 1 < completed write operation key=TIE8E7PAI3NC tag=89QS1Z0FLOP8 type=write size=292386 2023-01-24T10:52:24.828 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.827+0000 7fbe187898c0 1 > prepared part key=_multipart_Z4E99NU2FUTH.2~OR5ZCBMUTTY9.0 size=753571 2023-01-24T10:52:24.828 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.827+0000 7fbe187898c0 1 > prepared multipart upload key=Z4E99NU2FUTH upload=OR5ZCBMUTTY9 tag=5YUVWF0PFB8W type=write size=1802147 2023-01-24T10:52:24.832 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.831+0000 7fbe187898c0 1 < suggested remove operation key=1ZN0Y6AS7ISA tag=2DN7TGXUVI4T type=del 2023-01-24T10:52:24.834 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.833+0000 7fbe187898c0 1 > prepared operation key=RBS906ROG13P tag=08VTWR79DXVS type=del size=2889792 2023-01-24T10:52:24.836 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.834+0000 7fbe187898c0 1 < canceled operation key=Z8G7Y4RGVWR8 tag=ZXD1QXLYEDYM type=del size=2508175 2023-01-24T10:52:24.841 INFO:tasks.workunit.client.0.smithi078.stderr:2023-01-24T10:52:24.840+0000 7fbe187898c0 1 > prepared part key=_multipart_G