2023-01-24T19:01:53.831 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T19:01:53.832 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135639 --name yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi -- /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135639/orig.config.yaml 2023-01-24T19:01:53.877 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T19:01:53.947 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135639 branch: wip-yuri7-testing-2023-01-23-1532-quincy description: rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps} email: yweinstein@radhat.com first_in_suite: false job_id: '7135639' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 30 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri7-testing-2023-01-23-1532-quincy ceph: conf: global: mon client directed command retry: 5 mon election default strategy: 1 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async osd_pool_default_min_size: 2 osd_pool_default_size: 2 osd_recovery_max_active: 10 osd_recovery_max_single_start: 10 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd backoff on peering: true osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd heartbeat use min delay socket: true osd inject bad map crc probability: 0.1 osd max markdown count: 1000 osd mclock override recovery settings: true osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true osd_max_pg_log_entries: 3000 osd_min_pg_log_entries: 3000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(REQUEST_STUCK\) - failed to encode map - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - must scrub before tier agent can activate sha1: 49f8fb05584886826e8eade75f7105fba754560c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 49f8fb05584886826e8eade75f7105fba754560c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri7-testing-2023-01-23-1532-quincy sha1: 49f8fb05584886826e8eade75f7105fba754560c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b sha1: 49f8fb05584886826e8eade75f7105fba754560c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri7-testing-2023-01-23-1532-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49f8fb05584886826e8eade75f7105fba754560c targets: smithi161.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy1aqx1Mh+njlc7fQQEYxU88GQgNFDqWaTrIUB26LOyZlUOJXMTvLATxVditSbnexN+kyO3vZtRr2ZWK6GgOJhlgfUgKhDwRTAuUH8Wlehgs514F8n07nx5mrJKOj4rjypXiJjKV3FF1T8vh8yo2cpre2+pZXQIEDCdosyfcpfXyaEtPud1WD10u4xUAgY3SuC+66Tu/74e1O4waHyrA38O8Z151/EhD61J+frT9b2WWrI3wro0le4p6p/6PZ4z45fOaJRaNJ2pOQE+Yy+pwvrYQjQ7Y8Jy8CJYFpz9wAI5YypUquWvvv+xhYSIfOiX3gilcxqgqh/PMqCtcVcxgdKTaFp+6gKLiXtrlygXZEAFh8hUdIBGS0574MNrobpWG7Wl/lTLTaGpcz8QDz5h/mNdBJTrzaZenJWPA47bfdve6vo39C6zgjOKXyfMbv2xHmpHE4bSjY54SDwIr6l3aDt0imkT22hF+UMAIEFR1iZ7h0bAIoA3kCxMSNe9UbLEZE= smithi187.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbyX5Dx+eTTEL2QiGonHxt7lJUb0XN8i05S7zE3vxj0g1y+5Nr9ZP5VszpOBiDUeyBqeSINlkqtgOt7fr8BQeu9KQDDeKuOFvjci/NffpCtFgFFny8+Yb8ymxn4Iug63mYao/2z2viBivF9zNKAYfFBC7vLTpZ5QcY3R28v5ivLNaRsseziSvDtSzOZcm2BmEYcZJLuz9vvxo6Hbkh19WgSAaE6pLLJsMfNRJofuASFJ6urMOy0vnIed8RWCUSwSuVswZnvhhAKgBPLxwdOrlJ8Xodup7HvYXsteFyKBqHW0avZYGgx6GufC4IWE/j66FYAdXD97rKOEDtvMxUYiTe4ARyE0c73NO2/lq85jSxaTqeZym13Vks7OCdS1eAZNiSPfRKJc71+JTUmbGCEWBlKAxu5lQede6M+JfiN/t6rbhvthGSl/OH3MYsL5e1V7e5B4osxrBzThWPT4OXfQIVW4Ee47i/fRGSmR9M09yOm76MxJ1dItfdCHhl7Stv9/U= tasks: - install: null - ceph: conf: osd: debug monc: 20 - exec: mon.a: - while ! ceph balancer status ; do sleep 1 ; done - ceph balancer mode crush-compat - ceph balancer on - exec: client.0: - sudo ceph osd pool create base 4 - sudo ceph osd pool application enable base rados - sudo ceph osd pool create cache 4 - sudo ceph osd tier add base cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay base cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 3600 - sudo ceph osd pool set cache target_max_objects 250 - sudo ceph osd pool set cache min_read_recency_for_promote 2 - rados: clients: - client.0 objects: 500 op_weights: cache_evict: 50 cache_flush: 50 cache_try_flush: 50 copy_from: 50 delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 pools: - base teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_16:05:13 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T19:01:53.948 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa; will attempt to use it 2023-01-24T19:01:53.949 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa/tasks 2023-01-24T19:01:53.949 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T19:01:53.953 INFO:teuthology.task.internal:Checking packages... 2023-01-24T19:01:53.980 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '49f8fb05584886826e8eade75f7105fba754560c' 2023-01-24T19:01:53.981 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T19:01:53.981 INFO:teuthology.packaging:ref: None 2023-01-24T19:01:53.982 INFO:teuthology.packaging:tag: None 2023-01-24T19:01:53.982 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T19:01:53.982 INFO:teuthology.packaging:sha1: 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:01:53.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T19:01:54.214 INFO:teuthology.task.internal:Found packages for ceph version 17.2.5-727.g49f8fb05 2023-01-24T19:01:54.215 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T19:01:54.225 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T19:01:54.225 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T19:01:54.251 INFO:teuthology.task.internal:Saving configuration 2023-01-24T19:01:54.268 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T19:01:54.278 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T19:01:54.301 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 18:56:10.035178', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHywukFlK8evBNvBw0V46fxOdmo0gOsLFieiUHooSlKpudSM8wc8DJ9UbvY5RHkdfIFeWwP45JmDAuPEeiWCLkQ='} 2023-01-24T19:01:54.324 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi187.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135639', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 18:56:10.033979', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBO/TLo2TOiMqkP88BZsjIslYFesNWawhw35Zqte7OlAjEc9VLK2SOzEXnRN0wcyJL2rK7LeTSXChLdrOFqhN8gc='} 2023-01-24T19:01:54.324 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T19:01:54.334 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2023-01-24T19:01:54.334 INFO:teuthology.task.internal:roles: ubuntu@smithi187.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2023-01-24T19:01:54.334 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T19:01:54.402 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f40d5590670>, signals=[15]) 2023-01-24T19:01:54.403 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T19:01:54.413 INFO:teuthology.task.internal:Opening connections... 2023-01-24T19:01:54.413 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:01:54.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:01:54.484 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi187.front.sepia.ceph.com 2023-01-24T19:01:54.485 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:01:54.554 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T19:01:54.564 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2023-01-24T19:01:54.585 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2023-01-24T19:01:54.585 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2023-01-24T19:01:54.644 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="CentOS Stream" 2023-01-24T19:01:54.644 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="8" 2023-01-24T19:01:54.644 INFO:teuthology.orchestra.run.smithi161.stdout:ID="centos" 2023-01-24T19:01:54.644 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE="rhel fedora" 2023-01-24T19:01:54.645 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="8" 2023-01-24T19:01:54.645 INFO:teuthology.orchestra.run.smithi161.stdout:PLATFORM_ID="platform:el8" 2023-01-24T19:01:54.645 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-24T19:01:54.645 INFO:teuthology.orchestra.run.smithi161.stdout:ANSI_COLOR="0;31" 2023-01-24T19:01:54.645 INFO:teuthology.orchestra.run.smithi161.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-24T19:01:54.646 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://centos.org/" 2023-01-24T19:01:54.646 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T19:01:54.646 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T19:01:54.646 INFO:teuthology.orchestra.run.smithi161.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-24T19:01:54.647 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2023-01-24T19:01:54.664 DEBUG:teuthology.orchestra.run.smithi187:> uname -m 2023-01-24T19:01:54.684 INFO:teuthology.orchestra.run.smithi187.stdout:x86_64 2023-01-24T19:01:54.684 DEBUG:teuthology.orchestra.run.smithi187:> cat /etc/os-release 2023-01-24T19:01:54.741 INFO:teuthology.orchestra.run.smithi187.stdout:NAME="CentOS Stream" 2023-01-24T19:01:54.742 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION="8" 2023-01-24T19:01:54.742 INFO:teuthology.orchestra.run.smithi187.stdout:ID="centos" 2023-01-24T19:01:54.742 INFO:teuthology.orchestra.run.smithi187.stdout:ID_LIKE="rhel fedora" 2023-01-24T19:01:54.742 INFO:teuthology.orchestra.run.smithi187.stdout:VERSION_ID="8" 2023-01-24T19:01:54.743 INFO:teuthology.orchestra.run.smithi187.stdout:PLATFORM_ID="platform:el8" 2023-01-24T19:01:54.744 INFO:teuthology.orchestra.run.smithi187.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-24T19:01:54.744 INFO:teuthology.orchestra.run.smithi187.stdout:ANSI_COLOR="0;31" 2023-01-24T19:01:54.744 INFO:teuthology.orchestra.run.smithi187.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-24T19:01:54.744 INFO:teuthology.orchestra.run.smithi187.stdout:HOME_URL="https://centos.org/" 2023-01-24T19:01:54.744 INFO:teuthology.orchestra.run.smithi187.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T19:01:54.745 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T19:01:54.745 INFO:teuthology.orchestra.run.smithi187.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-24T19:01:54.745 INFO:teuthology.lock.ops:Updating smithi187.front.sepia.ceph.com on lock server 2023-01-24T19:01:54.759 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T19:01:54.772 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T19:01:54.783 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T19:01:54.783 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2023-01-24T19:01:54.786 DEBUG:teuthology.orchestra.run.smithi187:> test '!' -e /home/ubuntu/cephtest 2023-01-24T19:01:54.804 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T19:01:54.814 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T19:01:54.814 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2023-01-24T19:01:54.846 DEBUG:teuthology.orchestra.run.smithi187:> test -z $(ls -A /var/lib/ceph) 2023-01-24T19:01:54.877 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T19:01:54.933 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T19:01:54.951 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T19:01:54.952 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T19:01:54.952 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T19:01:54.953 DEBUG:teuthology.orchestra.run.smithi187:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T19:01:54.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:01:54.970 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2023-01-24T19:01:54.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:01:54.972 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-01-24T19:01:55.028 INFO:teuthology.orchestra.run.smithi187.stdout:4.18.0-408.el8.x86_64 2023-01-24T19:01:55.029 INFO:teuthology.task.kernel:Running kernel on smithi187: 4.18.0-408.el8.x86_64 2023-01-24T19:01:55.029 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum install -y kernel 2023-01-24T19:01:55.030 INFO:teuthology.orchestra.run.smithi161.stdout:4.18.0-408.el8.x86_64 2023-01-24T19:01:55.031 INFO:teuthology.task.kernel:Running kernel on smithi161: 4.18.0-408.el8.x86_64 2023-01-24T19:01:55.031 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum install -y kernel 2023-01-24T19:01:58.594 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - AppStream 35 MB/s | 27 MB 00:00 2023-01-24T19:01:59.238 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - AppStream 22 MB/s | 27 MB 00:01 2023-01-24T19:02:04.211 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - BaseOS 43 MB/s | 27 MB 00:00 2023-01-24T19:02:04.972 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - BaseOS 36 MB/s | 27 MB 00:00 2023-01-24T19:02:08.485 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - Extras 155 kB/s | 18 kB 00:00 2023-01-24T19:02:08.895 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - PowerTools 20 MB/s | 5.6 MB 00:00 2023-01-24T19:02:09.220 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - Extras 146 kB/s | 18 kB 00:00 2023-01-24T19:02:09.789 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - PowerTools 13 MB/s | 5.6 MB 00:00 2023-01-24T19:02:10.294 INFO:teuthology.orchestra.run.smithi187.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-01-24T19:02:10.996 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-01-24T19:02:12.936 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 5.3 MB/s | 13 MB 00:02 2023-01-24T19:02:13.297 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 6.2 MB/s | 13 MB 00:02 2023-01-24T19:02:16.236 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 560 kB/s | 24 kB 00:00 2023-01-24T19:02:16.663 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 282 kB/s | 24 kB 00:00 2023-01-24T19:02:19.376 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-24T19:02:19.376 INFO:teuthology.orchestra.run.smithi187.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-24T19:02:19.539 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:02:19.540 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:02:19.540 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:02:19.540 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:02:19.541 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:02:19.541 INFO:teuthology.orchestra.run.smithi187.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-24T19:02:19.541 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-24T19:02:19.541 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-24T19:02:19.541 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:02:19.542 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:02:19.542 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:02:19.542 INFO:teuthology.orchestra.run.smithi187.stdout:Install 3 Packages 2023-01-24T19:02:19.542 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:02:19.543 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 85 M 2023-01-24T19:02:19.543 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 94 M 2023-01-24T19:02:19.543 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:02:19.796 INFO:teuthology.orchestra.run.smithi187.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 36 MB/s | 9.2 MB 00:00 2023-01-24T19:02:19.820 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-24T19:02:19.820 INFO:teuthology.orchestra.run.smithi161.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-24T19:02:19.970 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:02:19.971 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:02:19.971 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:02:19.971 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:02:19.971 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:02:19.972 INFO:teuthology.orchestra.run.smithi161.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-24T19:02:19.972 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-24T19:02:19.972 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-24T19:02:19.972 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:02:19.972 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:02:19.973 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:02:19.973 INFO:teuthology.orchestra.run.smithi161.stdout:Install 3 Packages 2023-01-24T19:02:19.973 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:02:19.974 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 85 M 2023-01-24T19:02:19.974 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 94 M 2023-01-24T19:02:19.974 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:02:20.796 INFO:teuthology.orchestra.run.smithi187.stdout:(2/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 27 MB/s | 34 MB 00:01 2023-01-24T19:02:20.827 INFO:teuthology.orchestra.run.smithi161.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 11 MB/s | 9.2 MB 00:00 2023-01-24T19:02:21.171 INFO:teuthology.orchestra.run.smithi187.stdout:(3/3): kernel-core-4.18.0-448.el8.x86_64.rpm 25 MB/s | 41 MB 00:01 2023-01-24T19:02:21.174 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:02:21.174 INFO:teuthology.orchestra.run.smithi187.stdout:Total 52 MB/s | 85 MB 00:01 2023-01-24T19:02:21.623 INFO:teuthology.orchestra.run.smithi161.stdout:(2/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 21 MB/s | 34 MB 00:01 2023-01-24T19:02:21.807 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:02:22.007 INFO:teuthology.orchestra.run.smithi161.stdout:(3/3): kernel-core-4.18.0-448.el8.x86_64.rpm 20 MB/s | 41 MB 00:02 2023-01-24T19:02:22.010 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:02:22.010 INFO:teuthology.orchestra.run.smithi161.stdout:Total 41 MB/s | 85 MB 00:02 2023-01-24T19:02:22.200 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:02:22.200 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:02:22.644 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:02:23.056 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:02:23.057 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:02:24.273 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:02:24.296 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:02:25.167 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:02:25.208 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:02:27.011 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:02:27.598 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-24T19:02:27.951 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:02:28.632 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-24T19:02:31.381 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-24T19:02:31.797 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-24T19:02:32.554 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-24T19:02:33.006 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-24T19:02:35.117 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-24T19:02:35.372 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:36.331 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-24T19:02:36.616 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:57.045 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:57.070 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:57.486 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:57.486 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-24T19:02:57.486 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-24T19:02:58.067 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:58.067 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:02:58.068 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:02:58.068 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-24T19:02:58.068 INFO:teuthology.orchestra.run.smithi187.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-24T19:02:58.068 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:02:58.068 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:02:58.176 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel --last | head -n 1 2023-01-24T19:02:58.343 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:58.370 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:58.372 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-4.18.0-448.el8.x86_64 Tue 24 Jan 2023 07:02:35 PM UTC 2023-01-24T19:02:58.374 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-24T19:02:58.374 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-24T19:02:58.374 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-24T19:02:58.375 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-24T19:02:58.375 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-01-24T19:02:58.375 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi187.front.sepia.ceph.com, path=None, version=distro) 2023-01-24T19:02:58.375 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q kernel --last | head -n 1 2023-01-24T19:02:58.562 INFO:teuthology.orchestra.run.smithi187.stdout:kernel-4.18.0-448.el8.x86_64 Tue 24 Jan 2023 07:02:35 PM UTC 2023-01-24T19:02:58.564 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-24T19:02:58.565 DEBUG:teuthology.orchestra.run.smithi187:> sudo rpm -qi grub2-tools 2023-01-24T19:02:58.598 INFO:teuthology.orchestra.run.smithi187.stdout:Name : grub2-tools 2023-01-24T19:02:58.598 INFO:teuthology.orchestra.run.smithi187.stdout:Epoch : 1 2023-01-24T19:02:58.598 INFO:teuthology.orchestra.run.smithi187.stdout:Version : 2.02 2023-01-24T19:02:58.599 INFO:teuthology.orchestra.run.smithi187.stdout:Release : 129.el8 2023-01-24T19:02:58.599 INFO:teuthology.orchestra.run.smithi187.stdout:Architecture: x86_64 2023-01-24T19:02:58.599 INFO:teuthology.orchestra.run.smithi187.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-24T19:02:58.599 INFO:teuthology.orchestra.run.smithi187.stdout:Group : System Environment/Base 2023-01-24T19:02:58.599 INFO:teuthology.orchestra.run.smithi187.stdout:Size : 9265586 2023-01-24T19:02:58.600 INFO:teuthology.orchestra.run.smithi187.stdout:License : GPLv3+ 2023-01-24T19:02:58.600 INFO:teuthology.orchestra.run.smithi187.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-24T19:02:58.600 INFO:teuthology.orchestra.run.smithi187.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-24T19:02:58.600 INFO:teuthology.orchestra.run.smithi187.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-24T19:02:58.600 INFO:teuthology.orchestra.run.smithi187.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-24T19:02:58.601 INFO:teuthology.orchestra.run.smithi187.stdout:Relocations : (not relocatable) 2023-01-24T19:02:58.601 INFO:teuthology.orchestra.run.smithi187.stdout:Packager : CentOS BuildSystem 2023-01-24T19:02:58.601 INFO:teuthology.orchestra.run.smithi187.stdout:Vendor : CentOS 2023-01-24T19:02:58.601 INFO:teuthology.orchestra.run.smithi187.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-24T19:02:58.602 INFO:teuthology.orchestra.run.smithi187.stdout:Summary : Support tools for GRUB. 2023-01-24T19:02:58.602 INFO:teuthology.orchestra.run.smithi187.stdout:Description : 2023-01-24T19:02:58.602 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:02:58.602 INFO:teuthology.orchestra.run.smithi187.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-24T19:02:58.603 INFO:teuthology.orchestra.run.smithi187.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-24T19:02:58.603 INFO:teuthology.orchestra.run.smithi187.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-24T19:02:58.604 INFO:teuthology.orchestra.run.smithi187.stdout:hardware devices. 2023-01-24T19:02:58.604 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:02:58.604 INFO:teuthology.orchestra.run.smithi187.stdout:This subpackage provides tools for support of all platforms. 2023-01-24T19:02:58.606 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-24T19:02:58.606 INFO:teuthology.task.kernel:Updating grub on smithi187 to boot 4.18.0-448.el8.x86_64 2023-01-24T19:02:58.607 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-24T19:02:58.816 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:58.817 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-24T19:02:58.817 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-24T19:02:59.417 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-24T19:02:59.418 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:02:59.418 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:02:59.418 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-24T19:02:59.418 INFO:teuthology.orchestra.run.smithi161.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-24T19:02:59.418 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:02:59.419 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:02:59.530 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel --last | head -n 1 2023-01-24T19:02:59.727 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-4.18.0-448.el8.x86_64 Tue 24 Jan 2023 07:02:36 PM UTC 2023-01-24T19:02:59.728 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-24T19:02:59.729 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-24T19:02:59.729 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-24T19:02:59.729 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-24T19:02:59.729 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-01-24T19:02:59.729 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2023-01-24T19:02:59.730 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q kernel --last | head -n 1 2023-01-24T19:02:59.920 INFO:teuthology.orchestra.run.smithi161.stdout:kernel-4.18.0-448.el8.x86_64 Tue 24 Jan 2023 07:02:36 PM UTC 2023-01-24T19:02:59.922 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-24T19:02:59.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo rpm -qi grub2-tools 2023-01-24T19:02:59.955 INFO:teuthology.orchestra.run.smithi161.stdout:Name : grub2-tools 2023-01-24T19:02:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Epoch : 1 2023-01-24T19:02:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Version : 2.02 2023-01-24T19:02:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Release : 129.el8 2023-01-24T19:02:59.956 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: x86_64 2023-01-24T19:02:59.957 INFO:teuthology.orchestra.run.smithi161.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-24T19:02:59.957 INFO:teuthology.orchestra.run.smithi161.stdout:Group : System Environment/Base 2023-01-24T19:02:59.957 INFO:teuthology.orchestra.run.smithi161.stdout:Size : 9265586 2023-01-24T19:02:59.957 INFO:teuthology.orchestra.run.smithi161.stdout:License : GPLv3+ 2023-01-24T19:02:59.958 INFO:teuthology.orchestra.run.smithi161.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-24T19:02:59.958 INFO:teuthology.orchestra.run.smithi161.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-24T19:02:59.958 INFO:teuthology.orchestra.run.smithi161.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-24T19:02:59.958 INFO:teuthology.orchestra.run.smithi161.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-24T19:02:59.958 INFO:teuthology.orchestra.run.smithi161.stdout:Relocations : (not relocatable) 2023-01-24T19:02:59.959 INFO:teuthology.orchestra.run.smithi161.stdout:Packager : CentOS BuildSystem 2023-01-24T19:02:59.959 INFO:teuthology.orchestra.run.smithi161.stdout:Vendor : CentOS 2023-01-24T19:02:59.959 INFO:teuthology.orchestra.run.smithi161.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-24T19:02:59.959 INFO:teuthology.orchestra.run.smithi161.stdout:Summary : Support tools for GRUB. 2023-01-24T19:02:59.959 INFO:teuthology.orchestra.run.smithi161.stdout:Description : 2023-01-24T19:02:59.960 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:02:59.960 INFO:teuthology.orchestra.run.smithi161.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-24T19:02:59.960 INFO:teuthology.orchestra.run.smithi161.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-24T19:02:59.960 INFO:teuthology.orchestra.run.smithi161.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-24T19:02:59.960 INFO:teuthology.orchestra.run.smithi161.stdout:hardware devices. 2023-01-24T19:02:59.961 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:02:59.961 INFO:teuthology.orchestra.run.smithi161.stdout:This subpackage provides tools for support of all platforms. 2023-01-24T19:02:59.962 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-24T19:02:59.962 INFO:teuthology.task.kernel:Updating grub on smithi161 to boot 4.18.0-448.el8.x86_64 2023-01-24T19:02:59.962 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-24T19:03:00.370 INFO:teuthology.orchestra.run.smithi187.stderr:Generating grub configuration file ... 2023-01-24T19:03:01.733 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2023-01-24T19:03:02.216 INFO:teuthology.orchestra.run.smithi187.stderr:done 2023-01-24T19:03:02.217 DEBUG:teuthology.orchestra.run.smithi187:> mktemp 2023-01-24T19:03:02.257 INFO:teuthology.orchestra.run.smithi187.stdout:/tmp/tmp.7HTLDpPozq 2023-01-24T19:03:02.258 DEBUG:teuthology.orchestra.run.smithi187:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.7HTLDpPozq 2023-01-24T19:03:02.331 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0666 /tmp/tmp.7HTLDpPozq 2023-01-24T19:03:02.484 DEBUG:teuthology.orchestra.remote:smithi187:/tmp/tmp.7HTLDpPozq is 6KB 2023-01-24T19:03:02.532 DEBUG:teuthology.orchestra.run.smithi187:> rm -fr /tmp/tmp.7HTLDpPozq 2023-01-24T19:03:02.549 DEBUG:teuthology.orchestra.run.smithi187:> sudo /bin/ls /boot/loader/entries || true 2023-01-24T19:03:02.614 INFO:teuthology.orchestra.run.smithi187.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-24T19:03:02.614 INFO:teuthology.orchestra.run.smithi187.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-24T19:03:02.615 INFO:teuthology.orchestra.run.smithi187.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-24T19:03:02.615 INFO:teuthology.orchestra.run.smithi187.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-24T19:03:02.616 DEBUG:teuthology.orchestra.run.smithi187:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-24T19:03:02.788 DEBUG:teuthology.orchestra.run.smithi187:> sudo shutdown -r now 2023-01-24T19:03:03.629 INFO:teuthology.orchestra.run.smithi161.stderr:done 2023-01-24T19:03:03.630 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-01-24T19:03:03.654 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.oA7n511Li5 2023-01-24T19:03:03.654 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.oA7n511Li5 2023-01-24T19:03:03.736 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.oA7n511Li5 2023-01-24T19:03:03.894 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.oA7n511Li5 is 6KB 2023-01-24T19:03:03.942 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.oA7n511Li5 2023-01-24T19:03:03.959 DEBUG:teuthology.orchestra.run.smithi161:> sudo /bin/ls /boot/loader/entries || true 2023-01-24T19:03:04.034 INFO:teuthology.orchestra.run.smithi161.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-24T19:03:04.034 INFO:teuthology.orchestra.run.smithi161.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-24T19:03:04.034 INFO:teuthology.orchestra.run.smithi161.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-24T19:03:04.034 INFO:teuthology.orchestra.run.smithi161.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-24T19:03:04.035 DEBUG:teuthology.orchestra.run.smithi161:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-24T19:03:04.218 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2023-01-24T19:03:32.817 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-01-24T19:03:32.817 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-01-24T19:03:32.818 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:03:34.222 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-01-24T19:03:34.222 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-01-24T19:03:34.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:03:51.148 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.187 2023-01-24T19:03:52.680 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-01-24T19:04:21.177 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-01-24T19:04:21.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:04:22.683 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-01-24T19:04:22.684 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:04:36.732 DEBUG:teuthology.orchestra.run.smithi187:> true 2023-01-24T19:04:37.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-01-24T19:04:37.078 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-448.el8.x86_64"... 2023-01-24T19:04:37.078 DEBUG:teuthology.orchestra.run.smithi187:> uname -r 2023-01-24T19:04:37.096 INFO:teuthology.orchestra.run.smithi187.stdout:4.18.0-448.el8.x86_64 2023-01-24T19:04:37.096 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-24T19:04:37.096 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-24T19:04:37.097 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-24T19:04:38.098 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-01-24T19:04:38.098 DEBUG:teuthology.orchestra.run.smithi187:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T19:04:38.194 INFO:teuthology.orchestra.run.smithi187.stdout:ttyS1 2023-01-24T19:04:38.222 DEBUG:teuthology.parallel:result is None 2023-01-24T19:04:41.068 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-01-24T19:05:11.070 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-01-24T19:05:11.071 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:05:11.454 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-01-24T19:05:11.741 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-01-24T19:05:11.741 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-01-24T19:05:11.742 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-01-24T19:05:11.761 INFO:teuthology.orchestra.run.smithi161.stdout:4.18.0-448.el8.x86_64 2023-01-24T19:05:11.761 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-24T19:05:11.761 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-24T19:05:11.762 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-24T19:05:12.763 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T19:05:12.763 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T19:05:12.862 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2023-01-24T19:05:12.876 DEBUG:teuthology.parallel:result is None 2023-01-24T19:05:12.877 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T19:05:12.883 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T19:05:12.883 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T19:05:12.886 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T19:05:12.909 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T19:05:12.915 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T19:05:12.926 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T19:05:12.926 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T19:05:12.945 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T19:05:12.978 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T19:05:12.988 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T19:05:12.988 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T19:05:13.013 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T19:05:13.054 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:05:13.056 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:05:13.064 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:05:13.067 INFO:teuthology.orchestra.run.smithi187.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T19:05:13.069 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T19:05:13.076 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T19:05:13.076 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T19:05:13.108 DEBUG:teuthology.orchestra.run.smithi187:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T19:05:13.136 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T19:05:13.146 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T19:05:13.146 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T19:05:13.177 DEBUG:teuthology.orchestra.run.smithi187:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T19:05:13.195 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:05:13.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:05:13.328 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:05:13.409 DEBUG:teuthology.orchestra.run.smithi161:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:05:13.455 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:05:13.456 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T19:05:13.522 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:05:13.564 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T19:05:13.622 DEBUG:teuthology.orchestra.run.smithi187:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:05:13.706 DEBUG:teuthology.orchestra.run.smithi187:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T19:05:13.750 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:05:13.750 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T19:05:13.817 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2023-01-24T19:05:13.819 DEBUG:teuthology.orchestra.run.smithi187:> sudo service rsyslog restart 2023-01-24T19:05:13.881 INFO:teuthology.orchestra.run.smithi161.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T19:05:13.930 INFO:teuthology.orchestra.run.smithi187.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T19:05:14.353 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T19:05:14.361 INFO:teuthology.task.internal:Starting timer... 2023-01-24T19:05:14.362 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T19:05:14.378 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T19:05:14.387 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T19:05:14.387 DEBUG:teuthology.orchestra.run.smithi161:> sudo service auditd rotate 2023-01-24T19:05:14.469 INFO:teuthology.orchestra.run.smithi161.stdout:Rotating logs: 2023-01-24T19:05:14.471 DEBUG:teuthology.orchestra.run.smithi187:> sudo service auditd rotate 2023-01-24T19:05:14.550 INFO:teuthology.orchestra.run.smithi187.stdout:Rotating logs: 2023-01-24T19:05:14.552 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T19:05:14.552 DEBUG:teuthology.orchestra.run.smithi161:> /usr/sbin/getenforce 2023-01-24T19:05:14.573 INFO:teuthology.orchestra.run.smithi161.stdout:Permissive 2023-01-24T19:05:14.574 DEBUG:teuthology.orchestra.run.smithi187:> /usr/sbin/getenforce 2023-01-24T19:05:14.605 INFO:teuthology.orchestra.run.smithi187.stdout:Permissive 2023-01-24T19:05:14.606 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi161.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi187.front.sepia.ceph.com': 'permissive'} 2023-01-24T19:05:14.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T19:05:14.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:05:14.645 DEBUG:teuthology.orchestra.run.smithi187:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T19:05:14.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T19:05:14.674 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T19:05:14.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo /usr/sbin/setenforce permissive 2023-01-24T19:05:14.719 DEBUG:teuthology.orchestra.run.smithi187:> sudo /usr/sbin/setenforce permissive 2023-01-24T19:05:14.748 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T19:05:14.763 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-24T19:05:14.764 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T19:05:14.780 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T19:05:14.782 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi161.front.sepia.ceph.com,smithi187.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T19:11:47.002 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi161.front.sepia.ceph.com'), Remote(name='ubuntu@smithi187.front.sepia.ceph.com')] 2023-01-24T19:11:47.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-01-24T19:11:47.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:11:47.816 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-01-24T19:11:47.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi187.front.sepia.ceph.com' 2023-01-24T19:11:47.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi187.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T19:11:47.963 DEBUG:teuthology.orchestra.run.smithi187:> true 2023-01-24T19:11:48.031 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T19:11:48.041 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T19:11:48.042 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T19:11:48.043 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T19:11:48.045 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T19:11:48.045 DEBUG:teuthology.orchestra.run.smithi187:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T19:11:48.077 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T19:11:48.093 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T19:11:48.105 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T19:11:48.119 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T19:11:48.172 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: ntpd: command not found 2023-01-24T19:11:48.183 INFO:teuthology.orchestra.run.smithi187.stderr:sudo: ntpd: command not found 2023-01-24T19:11:48.184 INFO:teuthology.orchestra.run.smithi161.stdout:506 Cannot talk to daemon 2023-01-24T19:11:48.194 INFO:teuthology.orchestra.run.smithi187.stdout:506 Cannot talk to daemon 2023-01-24T19:11:48.199 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T19:11:48.208 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T19:11:48.212 INFO:teuthology.orchestra.run.smithi161.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T19:11:48.221 INFO:teuthology.orchestra.run.smithi187.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T19:11:48.281 INFO:teuthology.orchestra.run.smithi161.stderr:bash: ntpq: command not found 2023-01-24T19:11:48.283 INFO:teuthology.orchestra.run.smithi187.stderr:bash: ntpq: command not found 2023-01-24T19:11:48.285 INFO:teuthology.orchestra.run.smithi161.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T19:11:48.285 INFO:teuthology.orchestra.run.smithi161.stdout:=============================================================================== 2023-01-24T19:11:48.285 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.285 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.286 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.286 INFO:teuthology.orchestra.run.smithi161.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.287 INFO:teuthology.orchestra.run.smithi187.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T19:11:48.288 INFO:teuthology.orchestra.run.smithi187.stdout:=============================================================================== 2023-01-24T19:11:48.288 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.288 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.288 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.289 INFO:teuthology.orchestra.run.smithi187.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T19:11:48.289 INFO:teuthology.run_tasks:Running task install... 2023-01-24T19:11:48.297 DEBUG:teuthology.task.install:project ceph 2023-01-24T19:11:48.298 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T19:11:48.298 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'} 2023-01-24T19:11:48.298 INFO:teuthology.task.install:Using flavor: default 2023-01-24T19:11:48.304 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T19:11:48.305 INFO:teuthology.task.install:extra packages: [] 2023-01-24T19:11:48.305 DEBUG:teuthology.orchestra.run.smithi161:> sudo lsb_release -is 2023-01-24T19:11:48.364 INFO:teuthology.orchestra.run.smithi161.stdout:CentOSStream 2023-01-24T19:11:48.365 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T19:11:48.365 DEBUG:teuthology.orchestra.run.smithi187:> sudo lsb_release -is 2023-01-24T19:11:48.366 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T19:11:48.366 DEBUG:teuthology.orchestra.run.smithi161:> sudo lsb_release -is 2023-01-24T19:11:48.403 INFO:teuthology.orchestra.run.smithi187.stdout:CentOSStream 2023-01-24T19:11:48.404 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T19:11:48.404 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T19:11:48.405 DEBUG:teuthology.orchestra.run.smithi187:> sudo lsb_release -is 2023-01-24T19:11:48.441 INFO:teuthology.orchestra.run.smithi161.stdout:CentOSStream 2023-01-24T19:11:48.442 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T19:11:48.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:11:48.481 INFO:teuthology.orchestra.run.smithi187.stdout:CentOSStream 2023-01-24T19:11:48.491 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T19:11:48.491 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:11:48.658 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T19:11:48.659 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T19:11:48.698 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T19:11:48.698 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T19:11:48.834 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T19:11:48.835 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:11:48.835 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T19:11:48.863 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T19:11:48.864 DEBUG:teuthology.orchestra.run.smithi161:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T19:11:48.877 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T19:11:48.877 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:11:48.877 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T19:11:48.906 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T19:11:48.907 DEBUG:teuthology.orchestra.run.smithi187:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T19:11:48.932 DEBUG:teuthology.orchestra.run.smithi161:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T19:11:48.976 DEBUG:teuthology.orchestra.run.smithi187:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T19:11:49.011 DEBUG:teuthology.orchestra.run.smithi161:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T19:11:49.055 DEBUG:teuthology.orchestra.run.smithi187:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T19:11:49.077 INFO:teuthology.orchestra.run.smithi161.stdout:check_obsoletes = 1 2023-01-24T19:11:49.078 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum clean all 2023-01-24T19:11:49.122 INFO:teuthology.orchestra.run.smithi187.stdout:check_obsoletes = 1 2023-01-24T19:11:49.123 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum clean all 2023-01-24T19:11:49.328 INFO:teuthology.orchestra.run.smithi161.stdout:61 files removed 2023-01-24T19:11:49.358 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-radosgw 2023-01-24T19:11:49.371 INFO:teuthology.orchestra.run.smithi187.stdout:61 files removed 2023-01-24T19:11:49.398 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-radosgw 2023-01-24T19:11:50.265 INFO:teuthology.orchestra.run.smithi161.stdout:ceph packages for x86_64 157 kB/s | 78 kB 00:00 2023-01-24T19:11:50.465 INFO:teuthology.orchestra.run.smithi187.stdout:ceph packages for x86_64 119 kB/s | 78 kB 00:00 2023-01-24T19:11:50.682 INFO:teuthology.orchestra.run.smithi161.stdout:ceph noarch packages 24 kB/s | 9.2 kB 00:00 2023-01-24T19:11:50.858 INFO:teuthology.orchestra.run.smithi187.stdout:ceph noarch packages 26 kB/s | 9.2 kB 00:00 2023-01-24T19:11:51.091 INFO:teuthology.orchestra.run.smithi161.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-01-24T19:11:51.275 INFO:teuthology.orchestra.run.smithi187.stdout:ceph source packages 3.7 kB/s | 1.5 kB 00:00 2023-01-24T19:11:51.894 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - AppStream 35 MB/s | 27 MB 00:00 2023-01-24T19:11:52.194 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - AppStream 31 MB/s | 27 MB 00:00 2023-01-24T19:11:57.552 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - BaseOS 41 MB/s | 27 MB 00:00 2023-01-24T19:11:58.552 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - BaseOS 20 MB/s | 27 MB 00:01 2023-01-24T19:12:01.901 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - Extras 117 kB/s | 18 kB 00:00 2023-01-24T19:12:02.545 INFO:teuthology.orchestra.run.smithi161.stdout:CentOS-8 - PowerTools 11 MB/s | 5.6 MB 00:00 2023-01-24T19:12:02.901 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - Extras 107 kB/s | 18 kB 00:00 2023-01-24T19:12:03.553 INFO:teuthology.orchestra.run.smithi187.stdout:CentOS-8 - PowerTools 11 MB/s | 5.6 MB 00:00 2023-01-24T19:12:03.794 INFO:teuthology.orchestra.run.smithi161.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-01-24T19:12:04.827 INFO:teuthology.orchestra.run.smithi187.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-01-24T19:12:09.961 INFO:teuthology.orchestra.run.smithi187.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 13 MB 00:04 2023-01-24T19:12:10.069 INFO:teuthology.orchestra.run.smithi161.stdout:Extra Packages for Enterprise Linux 2.2 MB/s | 13 MB 00:06 2023-01-24T19:12:13.278 INFO:teuthology.orchestra.run.smithi187.stdout:lab-extras 325 kB/s | 24 kB 00:00 2023-01-24T19:12:13.361 INFO:teuthology.orchestra.run.smithi161.stdout:lab-extras 424 kB/s | 24 kB 00:00 2023-01-24T19:12:16.679 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:12:16.681 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:12:16.682 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:12:16.682 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:12:16.682 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:12:16.682 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:12:16.683 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrading: 2023-01-24T19:12:16.683 INFO:teuthology.orchestra.run.smithi187.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:12:16.683 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T19:12:16.683 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:12:16.683 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T19:12:16.684 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T19:12:16.684 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T19:12:16.684 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T19:12:16.684 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T19:12:16.684 INFO:teuthology.orchestra.run.smithi187.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T19:12:16.685 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-24T19:12:16.685 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T19:12:16.685 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-24T19:12:16.685 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T19:12:16.685 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T19:12:16.686 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-24T19:12:16.686 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T19:12:16.686 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T19:12:16.686 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T19:12:16.686 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-24T19:12:16.687 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T19:12:16.687 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T19:12:16.687 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T19:12:16.687 INFO:teuthology.orchestra.run.smithi187.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T19:12:16.687 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:12:16.687 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:12:16.688 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:12:16.688 INFO:teuthology.orchestra.run.smithi187.stdout:Install 21 Packages 2023-01-24T19:12:16.688 INFO:teuthology.orchestra.run.smithi187.stdout:Upgrade 2 Packages 2023-01-24T19:12:16.688 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:12:16.689 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 63 M 2023-01-24T19:12:16.689 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:12:16.848 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:12:16.851 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:12:16.851 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:12:16.851 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:12:16.851 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:12:16.851 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:12:16.852 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrading: 2023-01-24T19:12:16.852 INFO:teuthology.orchestra.run.smithi161.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:12:16.852 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T19:12:16.852 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:12:16.853 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T19:12:16.853 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T19:12:16.853 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T19:12:16.853 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T19:12:16.853 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T19:12:16.854 INFO:teuthology.orchestra.run.smithi161.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T19:12:16.854 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-24T19:12:16.854 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T19:12:16.854 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-24T19:12:16.854 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T19:12:16.855 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T19:12:16.855 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-24T19:12:16.855 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T19:12:16.855 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T19:12:16.855 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T19:12:16.856 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-24T19:12:16.856 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T19:12:16.856 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T19:12:16.856 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T19:12:16.856 INFO:teuthology.orchestra.run.smithi161.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T19:12:16.857 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:12:16.857 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:12:16.857 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:12:16.857 INFO:teuthology.orchestra.run.smithi161.stdout:Install 21 Packages 2023-01-24T19:12:16.857 INFO:teuthology.orchestra.run.smithi161.stdout:Upgrade 2 Packages 2023-01-24T19:12:16.857 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:12:16.858 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 63 M 2023-01-24T19:12:16.858 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:12:17.687 INFO:teuthology.orchestra.run.smithi187.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 5.1 MB/s | 5.1 MB 00:01 2023-01-24T19:12:17.745 INFO:teuthology.orchestra.run.smithi187.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 418 kB/s | 24 kB 00:00 2023-01-24T19:12:18.054 INFO:teuthology.orchestra.run.smithi187.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 2.5 MB/s | 793 kB 00:00 2023-01-24T19:12:18.204 INFO:teuthology.orchestra.run.smithi187.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 3.3 MB/s | 506 kB 00:00 2023-01-24T19:12:19.096 INFO:teuthology.orchestra.run.smithi187.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 6.2 MB/s | 5.5 MB 00:00 2023-01-24T19:12:19.171 INFO:teuthology.orchestra.run.smithi187.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 596 kB/s | 45 kB 00:00 2023-01-24T19:12:19.246 INFO:teuthology.orchestra.run.smithi187.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 1.4 MB/s | 108 kB 00:00 2023-01-24T19:12:19.330 INFO:teuthology.orchestra.run.smithi187.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 2.2 MB/s | 190 kB 00:00 2023-01-24T19:12:19.422 INFO:teuthology.orchestra.run.smithi187.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 4.0 MB/s | 370 kB 00:00 2023-01-24T19:12:19.514 INFO:teuthology.orchestra.run.smithi187.stdout:(10/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 3.8 MB/s | 357 kB 00:00 2023-01-24T19:12:19.598 INFO:teuthology.orchestra.run.smithi187.stdout:(11/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 1.2 MB/s | 107 kB 00:00 2023-01-24T19:12:19.656 INFO:teuthology.orchestra.run.smithi187.stdout:(12/23): librdkafka-0.11.4-3.el8.x86_64.rpm 5.9 MB/s | 354 kB 00:00 2023-01-24T19:12:19.681 INFO:teuthology.orchestra.run.smithi187.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2023-01-24T19:12:19.707 INFO:teuthology.orchestra.run.smithi187.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.7 MB/s | 44 kB 00:00 2023-01-24T19:12:19.732 INFO:teuthology.orchestra.run.smithi187.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-01-24T19:12:19.824 INFO:teuthology.orchestra.run.smithi161.stdout:(1/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8.x 5.6 MB/s | 17 MB 00:02 2023-01-24T19:12:19.836 INFO:teuthology.orchestra.run.smithi187.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Status code: 503 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm (IP: 152.19.134.145) 2023-01-24T19:12:19.866 INFO:teuthology.orchestra.run.smithi187.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 2.2 MB/s | 306 kB 00:00 2023-01-24T19:12:19.912 INFO:teuthology.orchestra.run.smithi161.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 279 kB/s | 24 kB 00:00 2023-01-24T19:12:19.987 INFO:teuthology.orchestra.run.smithi187.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Status code: 503 for http://download-ib01.fedoraproject.org/pub/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm (IP: 152.19.134.145) 2023-01-24T19:12:20.016 INFO:teuthology.orchestra.run.smithi187.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 392 kB/s | 59 kB 00:00 2023-01-24T19:12:20.020 INFO:teuthology.orchestra.run.smithi161.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 7.2 MB/s | 793 kB 00:00 2023-01-24T19:12:20.154 INFO:teuthology.orchestra.run.smithi161.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 3.7 MB/s | 506 kB 00:00 2023-01-24T19:12:20.679 INFO:teuthology.orchestra.run.smithi161.stdout:(5/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 11 MB/s | 5.5 MB 00:00 2023-01-24T19:12:20.733 INFO:teuthology.orchestra.run.smithi187.stdout:(18/23): libunwind-1.3.1-3.el8.x86_64.rpm 105 kB/s | 75 kB 00:00 2023-01-24T19:12:20.797 INFO:teuthology.orchestra.run.smithi161.stdout:(6/23): python3-ceph-argparse-17.2.5-727.g49f8f 381 kB/s | 45 kB 00:00 2023-01-24T19:12:20.872 INFO:teuthology.orchestra.run.smithi161.stdout:(7/23): python3-ceph-common-17.2.5-727.g49f8fb0 1.4 MB/s | 108 kB 00:00 2023-01-24T19:12:20.930 INFO:teuthology.orchestra.run.smithi161.stdout:(8/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 3.2 MB/s | 190 kB 00:00 2023-01-24T19:12:21.006 INFO:teuthology.orchestra.run.smithi161.stdout:(9/23): python3-rados-17.2.5-727.g49f8fb05.el8. 4.8 MB/s | 370 kB 00:00 2023-01-24T19:12:21.098 INFO:teuthology.orchestra.run.smithi161.stdout:(10/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 3.8 MB/s | 357 kB 00:00 2023-01-24T19:12:21.156 INFO:teuthology.orchestra.run.smithi161.stdout:(11/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 1.8 MB/s | 107 kB 00:00 2023-01-24T19:12:21.257 INFO:teuthology.orchestra.run.smithi161.stdout:(12/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86 1.2 MB/s | 5.1 MB 00:04 2023-01-24T19:12:21.298 INFO:teuthology.orchestra.run.smithi161.stdout:(13/23): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2023-01-24T19:12:21.315 INFO:teuthology.orchestra.run.smithi161.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 2.9 MB/s | 44 kB 00:00 2023-01-24T19:12:21.341 INFO:teuthology.orchestra.run.smithi161.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-01-24T19:12:21.388 INFO:teuthology.orchestra.run.smithi161.stdout:(16/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2023-01-24T19:12:21.534 INFO:teuthology.orchestra.run.smithi187.stdout:(19/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 4.8 MB/s | 23 MB 00:04 2023-01-24T19:12:21.541 INFO:teuthology.orchestra.run.smithi161.stdout:(17/23): liboath-2.6.2-3.el8.x86_64.rpm 384 kB/s | 59 kB 00:00 2023-01-24T19:12:21.600 INFO:teuthology.orchestra.run.smithi161.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-01-24T19:12:21.625 INFO:teuthology.orchestra.run.smithi161.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 901 kB/s | 75 kB 00:00 2023-01-24T19:12:21.892 INFO:teuthology.orchestra.run.smithi187.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 1.5 MB/s | 1.7 MB 00:01 2023-01-24T19:12:21.942 INFO:teuthology.orchestra.run.smithi161.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 5.1 MB/s | 1.7 MB 00:00 2023-01-24T19:12:21.976 INFO:teuthology.orchestra.run.smithi187.stdout:(21/23): librados2-17.2.5-727.g49f8fb05.el8.x86 7.8 MB/s | 3.5 MB 00:00 2023-01-24T19:12:22.252 INFO:teuthology.orchestra.run.smithi161.stdout:(21/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 12 MB/s | 3.6 MB 00:00 2023-01-24T19:12:22.967 INFO:teuthology.orchestra.run.smithi187.stdout:(22/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 3.3 MB/s | 3.6 MB 00:01 2023-01-24T19:12:23.426 INFO:teuthology.orchestra.run.smithi161.stdout:(22/23): librados2-17.2.5-727.g49f8fb05.el8.x86 1.9 MB/s | 3.5 MB 00:01 2023-01-24T19:12:23.584 INFO:teuthology.orchestra.run.smithi187.stdout:(23/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8. 2.4 MB/s | 17 MB 00:06 2023-01-24T19:12:23.588 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:12:23.588 INFO:teuthology.orchestra.run.smithi187.stdout:Total 9.1 MB/s | 63 MB 00:06 2023-01-24T19:12:23.631 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:12:23.676 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:12:23.676 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:12:24.311 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:12:24.315 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:12:25.226 INFO:teuthology.orchestra.run.smithi161.stdout:(23/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 2.8 MB/s | 23 MB 00:08 2023-01-24T19:12:25.230 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:12:25.230 INFO:teuthology.orchestra.run.smithi161.stdout:Total 7.5 MB/s | 63 MB 00:08 2023-01-24T19:12:25.273 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:12:25.320 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:12:25.321 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:12:25.375 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:12:25.624 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-01-24T19:12:25.825 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-24T19:12:25.947 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:12:25.951 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:12:26.207 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-24T19:12:26.425 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:12:26.517 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:12:26.808 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T19:12:27.008 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:12:27.107 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:12:27.343 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-01-24T19:12:27.405 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:12:27.512 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-24T19:12:27.607 INFO:teuthology.orchestra.run.smithi187.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:12:27.852 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:12:27.899 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-24T19:12:28.046 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-01-24T19:12:28.139 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:12:28.231 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T19:12:28.234 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:12:28.383 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-24T19:12:28.537 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T19:12:28.754 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:12:28.948 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-24T19:12:29.116 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:12:29.144 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:12:29.177 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:12:29.310 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T19:12:29.446 INFO:teuthology.orchestra.run.smithi161.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:12:29.527 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:12:29.655 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:12:29.784 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:12:29.834 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-01-24T19:12:30.028 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T19:12:30.035 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T19:12:30.170 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-24T19:12:30.198 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:12:30.267 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:12:30.747 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T19:12:30.755 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-24T19:12:30.953 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:12:30.974 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2023-01-24T19:12:31.019 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:12:31.159 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T19:12:31.168 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-24T19:12:31.373 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:12:31.391 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T19:12:31.630 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:12:31.900 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T19:12:32.053 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:12:32.147 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T19:12:32.348 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T19:12:32.620 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 17/25 2023-01-24T19:12:32.844 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-24T19:12:33.028 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T19:12:35.076 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:12:35.307 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:12:35.882 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:12:36.363 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:12:36.457 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:12:36.639 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:12:36.768 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:12:37.036 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:12:37.633 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T19:12:37.958 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:12:38.044 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T19:12:38.261 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:12:55.577 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:12:55.577 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /sys 2023-01-24T19:12:55.577 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /proc 2023-01-24T19:12:55.577 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /dev 2023-01-24T19:12:55.578 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /run 2023-01-24T19:12:55.578 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /mnt 2023-01-24T19:12:55.578 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /var/tmp 2023-01-24T19:12:55.578 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /home 2023-01-24T19:12:55.578 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /tmp 2023-01-24T19:12:55.579 INFO:teuthology.orchestra.run.smithi187.stdout:skipping the directory /dev 2023-01-24T19:12:55.579 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:12:57.343 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:12:57.377 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:12:57.377 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /sys 2023-01-24T19:12:57.377 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /proc 2023-01-24T19:12:57.377 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-01-24T19:12:57.378 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /run 2023-01-24T19:12:57.378 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /mnt 2023-01-24T19:12:57.378 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /var/tmp 2023-01-24T19:12:57.378 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /home 2023-01-24T19:12:57.378 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /tmp 2023-01-24T19:12:57.379 INFO:teuthology.orchestra.run.smithi161.stdout:skipping the directory /dev 2023-01-24T19:12:57.379 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:12:57.415 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:12:57.415 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:12:57.646 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:12:57.647 INFO:teuthology.orchestra.run.smithi187.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:12:58.362 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:12:58.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T19:12:58.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:12:58.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:12:58.363 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T19:12:58.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:12:58.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:12:58.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T19:12:58.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T19:12:58.364 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T19:12:58.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:12:58.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T19:12:58.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:12:58.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:12:58.365 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-24T19:12:58.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-24T19:12:58.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T19:12:58.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-24T19:12:58.366 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T19:12:58.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-24T19:12:58.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-24T19:12:58.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-24T19:12:58.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:12:58.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T19:12:58.367 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T19:12:59.138 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:12:59.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:12:59.185 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:12:59.185 INFO:teuthology.orchestra.run.smithi187.stdout:Upgraded: 2023-01-24T19:12:59.185 INFO:teuthology.orchestra.run.smithi187.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.186 INFO:teuthology.orchestra.run.smithi187.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.186 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:12:59.186 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.186 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.186 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.187 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.187 INFO:teuthology.orchestra.run.smithi187.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T19:12:59.187 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.187 INFO:teuthology.orchestra.run.smithi187.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T19:12:59.187 INFO:teuthology.orchestra.run.smithi187.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T19:12:59.188 INFO:teuthology.orchestra.run.smithi187.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.188 INFO:teuthology.orchestra.run.smithi187.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-24T19:12:59.188 INFO:teuthology.orchestra.run.smithi187.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.188 INFO:teuthology.orchestra.run.smithi187.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T19:12:59.188 INFO:teuthology.orchestra.run.smithi187.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T19:12:59.189 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.189 INFO:teuthology.orchestra.run.smithi187.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.189 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.189 INFO:teuthology.orchestra.run.smithi187.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T19:12:59.189 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.190 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.190 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:12:59.190 INFO:teuthology.orchestra.run.smithi187.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T19:12:59.190 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:12:59.190 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:12:59.197 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T19:12:59.197 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:12:59.426 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T19:12:59.427 INFO:teuthology.orchestra.run.smithi161.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:12:59.469 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-test 2023-01-24T19:13:00.188 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:13:00.189 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T19:13:00.189 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T19:13:00.189 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T19:13:00.189 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T19:13:00.190 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T19:13:00.190 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T19:13:00.190 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T19:13:00.190 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T19:13:00.190 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T19:13:00.190 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T19:13:00.191 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T19:13:00.191 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T19:13:00.191 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T19:13:00.191 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-24T19:13:00.191 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-24T19:13:00.192 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T19:13:00.192 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-24T19:13:00.192 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T19:13:00.192 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-24T19:13:00.192 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-24T19:13:00.193 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-24T19:13:00.193 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T19:13:00.193 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T19:13:00.194 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T19:13:00.194 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:01.079 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T19:13:01.079 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:01.079 INFO:teuthology.orchestra.run.smithi161.stdout:Upgraded: 2023-01-24T19:13:01.080 INFO:teuthology.orchestra.run.smithi161.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.080 INFO:teuthology.orchestra.run.smithi161.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.080 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:13:01.080 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.081 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.081 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.081 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.081 INFO:teuthology.orchestra.run.smithi161.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T19:13:01.081 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.082 INFO:teuthology.orchestra.run.smithi161.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T19:13:01.082 INFO:teuthology.orchestra.run.smithi161.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T19:13:01.082 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.082 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-24T19:13:01.082 INFO:teuthology.orchestra.run.smithi161.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.083 INFO:teuthology.orchestra.run.smithi161.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T19:13:01.083 INFO:teuthology.orchestra.run.smithi161.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T19:13:01.083 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.083 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.083 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.084 INFO:teuthology.orchestra.run.smithi161.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T19:13:01.084 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.084 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.084 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:01.084 INFO:teuthology.orchestra.run.smithi161.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T19:13:01.085 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:01.085 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:13:01.338 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:13:01.338 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:01.339 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:13:01.339 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:01.339 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:13:01.339 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T19:13:01.340 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:13:01.340 INFO:teuthology.orchestra.run.smithi187.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-24T19:13:01.340 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T19:13:01.340 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-24T19:13:01.340 INFO:teuthology.orchestra.run.smithi187.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-24T19:13:01.341 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-24T19:13:01.341 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:01.341 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:13:01.341 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:01.341 INFO:teuthology.orchestra.run.smithi187.stdout:Install 6 Packages 2023-01-24T19:13:01.342 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:01.342 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 53 M 2023-01-24T19:13:01.343 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 233 M 2023-01-24T19:13:01.343 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:13:01.401 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-test 2023-01-24T19:13:01.441 INFO:teuthology.orchestra.run.smithi187.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2023-01-24T19:13:01.474 INFO:teuthology.orchestra.run.smithi187.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.3 MB/s | 187 kB 00:00 2023-01-24T19:13:01.525 INFO:teuthology.orchestra.run.smithi187.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-01-24T19:13:01.550 INFO:teuthology.orchestra.run.smithi187.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2023-01-24T19:13:01.784 INFO:teuthology.orchestra.run.smithi187.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 393 kB/s | 174 kB 00:00 2023-01-24T19:13:02.129 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:00:49 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:03.290 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:13:03.291 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:03.291 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:13:03.291 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:03.291 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:13:03.291 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T19:13:03.292 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:13:03.292 INFO:teuthology.orchestra.run.smithi161.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-24T19:13:03.292 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T19:13:03.292 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-24T19:13:03.293 INFO:teuthology.orchestra.run.smithi161.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-24T19:13:03.293 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-24T19:13:03.293 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:03.293 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:13:03.293 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:03.294 INFO:teuthology.orchestra.run.smithi161.stdout:Install 6 Packages 2023-01-24T19:13:03.294 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:03.295 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 53 M 2023-01-24T19:13:03.295 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 233 M 2023-01-24T19:13:03.295 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:13:03.389 INFO:teuthology.orchestra.run.smithi161.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.1 MB/s | 202 kB 00:00 2023-01-24T19:13:03.414 INFO:teuthology.orchestra.run.smithi161.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 11 MB/s | 187 kB 00:00 2023-01-24T19:13:03.440 INFO:teuthology.orchestra.run.smithi161.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 13 MB/s | 323 kB 00:00 2023-01-24T19:13:03.465 INFO:teuthology.orchestra.run.smithi161.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 71 kB 00:00 2023-01-24T19:13:03.757 INFO:teuthology.orchestra.run.smithi161.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 375 kB/s | 174 kB 00:00 2023-01-24T19:13:14.582 INFO:teuthology.orchestra.run.smithi187.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 4.0 MB/s | 52 MB 00:13 2023-01-24T19:13:14.584 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:14.585 INFO:teuthology.orchestra.run.smithi187.stdout:Total 4.0 MB/s | 53 MB 00:13 2023-01-24T19:13:14.627 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:13:14.648 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:13:14.649 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:13:14.968 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:13:14.971 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:13:15.463 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:13:15.724 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-24T19:13:15.942 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-24T19:13:16.107 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-24T19:13:16.187 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-24T19:13:16.380 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-24T19:13:16.549 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:13:19.238 INFO:teuthology.orchestra.run.smithi161.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 3.3 MB/s | 52 MB 00:15 2023-01-24T19:13:19.241 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:19.242 INFO:teuthology.orchestra.run.smithi161.stdout:Total 3.3 MB/s | 53 MB 00:15 2023-01-24T19:13:19.284 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:13:19.304 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:13:19.305 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:13:19.620 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:13:19.623 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:13:20.149 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:13:20.398 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-24T19:13:20.599 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-24T19:13:20.787 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-24T19:13:20.868 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-24T19:13:21.102 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-24T19:13:21.271 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:13:21.571 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:13:21.886 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:13:22.183 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:13:22.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T19:13:22.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T19:13:22.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-24T19:13:22.184 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-24T19:13:22.185 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-24T19:13:22.744 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-24T19:13:22.744 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:22.744 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:13:22.745 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:22.745 INFO:teuthology.orchestra.run.smithi187.stdout: jq-1.6-6.el8.x86_64 2023-01-24T19:13:22.745 INFO:teuthology.orchestra.run.smithi187.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:22.745 INFO:teuthology.orchestra.run.smithi187.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T19:13:22.745 INFO:teuthology.orchestra.run.smithi187.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-24T19:13:22.746 INFO:teuthology.orchestra.run.smithi187.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T19:13:22.746 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:22.746 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:13:22.940 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph 2023-01-24T19:13:23.668 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:10 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:24.812 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:13:24.815 INFO:teuthology.orchestra.run.smithi187.stdout:====================================================================================== 2023-01-24T19:13:24.816 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:13:24.816 INFO:teuthology.orchestra.run.smithi187.stdout:====================================================================================== 2023-01-24T19:13:24.816 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:13:24.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T19:13:24.817 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:13:24.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T19:13:24.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T19:13:24.817 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T19:13:24.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T19:13:24.818 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:13:24.818 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T19:13:24.818 INFO:teuthology.orchestra.run.smithi187.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T19:13:24.818 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-24T19:13:24.819 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T19:13:24.819 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T19:13:24.819 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-24T19:13:24.819 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T19:13:24.820 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T19:13:24.820 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T19:13:24.820 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-24T19:13:24.820 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-24T19:13:24.820 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T19:13:24.821 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T19:13:24.821 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T19:13:24.821 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-24T19:13:24.821 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-24T19:13:24.822 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-24T19:13:24.822 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T19:13:24.822 INFO:teuthology.orchestra.run.smithi187.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T19:13:24.822 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T19:13:24.822 INFO:teuthology.orchestra.run.smithi187.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T19:13:24.823 INFO:teuthology.orchestra.run.smithi187.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T19:13:24.823 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T19:13:24.823 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T19:13:24.823 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-24T19:13:24.823 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T19:13:24.824 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:24.824 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:13:24.824 INFO:teuthology.orchestra.run.smithi187.stdout:====================================================================================== 2023-01-24T19:13:24.824 INFO:teuthology.orchestra.run.smithi187.stdout:Install 32 Packages 2023-01-24T19:13:24.825 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:24.826 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 28 M 2023-01-24T19:13:24.826 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 110 M 2023-01-24T19:13:24.826 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:13:25.059 INFO:teuthology.orchestra.run.smithi187.stdout:(1/32): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 27 kB/s | 6.4 kB 00:00 2023-01-24T19:13:25.459 INFO:teuthology.orchestra.run.smithi187.stdout:(2/32): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 3.6 MB/s | 2.3 MB 00:00 2023-01-24T19:13:25.501 INFO:teuthology.orchestra.run.smithi187.stdout:(3/32): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 2.3 MB/s | 1.6 MB 00:00 2023-01-24T19:13:25.585 INFO:teuthology.orchestra.run.smithi187.stdout:(4/32): ceph-mgr-modules-core-17.2.5-727.g49f8f 2.7 MB/s | 233 kB 00:00 2023-01-24T19:13:25.643 INFO:teuthology.orchestra.run.smithi187.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-24T19:13:25.668 INFO:teuthology.orchestra.run.smithi187.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-01-24T19:13:25.694 INFO:teuthology.orchestra.run.smithi187.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.1 MB/s | 103 kB 00:00 2023-01-24T19:13:25.719 INFO:teuthology.orchestra.run.smithi187.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-01-24T19:13:25.761 INFO:teuthology.orchestra.run.smithi187.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 11 MB/s | 457 kB 00:00 2023-01-24T19:13:25.786 INFO:teuthology.orchestra.run.smithi187.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2023-01-24T19:13:25.820 INFO:teuthology.orchestra.run.smithi187.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-24T19:13:25.853 INFO:teuthology.orchestra.run.smithi187.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 3.2 MB/s | 109 kB 00:00 2023-01-24T19:13:26.104 INFO:teuthology.orchestra.run.smithi187.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 174 kB/s | 44 kB 00:00 2023-01-24T19:13:26.291 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T19:13:26.321 INFO:teuthology.orchestra.run.smithi187.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 855 kB/s | 185 kB 00:00 2023-01-24T19:13:26.505 INFO:teuthology.orchestra.run.smithi187.stdout:(15/32): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 16 MB/s | 17 MB 00:01 2023-01-24T19:13:26.530 INFO:teuthology.orchestra.run.smithi187.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 827 kB/s | 173 kB 00:00 2023-01-24T19:13:26.583 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:13:26.589 INFO:teuthology.orchestra.run.smithi187.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 190 kB/s | 11 kB 00:00 2023-01-24T19:13:26.656 INFO:teuthology.orchestra.run.smithi187.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 264 kB/s | 18 kB 00:00 2023-01-24T19:13:26.723 INFO:teuthology.orchestra.run.smithi187.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 729 kB/s | 49 kB 00:00 2023-01-24T19:13:26.789 INFO:teuthology.orchestra.run.smithi187.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 883 kB/s | 59 kB 00:00 2023-01-24T19:13:26.873 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T19:13:26.874 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T19:13:26.874 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T19:13:26.874 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-24T19:13:26.874 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-24T19:13:26.875 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-24T19:13:26.898 INFO:teuthology.orchestra.run.smithi187.stdout:(21/32): python3-cherrypy-18.4.0-1.el8.noarch.r 978 kB/s | 384 kB 00:00 2023-01-24T19:13:26.932 INFO:teuthology.orchestra.run.smithi187.stdout:(22/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.9 MB/s | 283 kB 00:00 2023-01-24T19:13:26.949 INFO:teuthology.orchestra.run.smithi187.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 310 kB/s | 16 kB 00:00 2023-01-24T19:13:26.982 INFO:teuthology.orchestra.run.smithi187.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 371 kB/s | 19 kB 00:00 2023-01-24T19:13:27.008 INFO:teuthology.orchestra.run.smithi187.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 407 kB/s | 24 kB 00:00 2023-01-24T19:13:27.041 INFO:teuthology.orchestra.run.smithi187.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 499 kB/s | 29 kB 00:00 2023-01-24T19:13:27.067 INFO:teuthology.orchestra.run.smithi187.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 470 kB/s | 27 kB 00:00 2023-01-24T19:13:27.150 INFO:teuthology.orchestra.run.smithi187.stdout:(28/32): python3-waitress-1.2.1-2.el8.1.noarch. 2.2 MB/s | 241 kB 00:00 2023-01-24T19:13:27.184 INFO:teuthology.orchestra.run.smithi187.stdout:(29/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.1 MB/s | 251 kB 00:00 2023-01-24T19:13:27.209 INFO:teuthology.orchestra.run.smithi187.stdout:(30/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.4 MB/s | 85 kB 00:00 2023-01-24T19:13:27.234 INFO:teuthology.orchestra.run.smithi187.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 457 kB/s | 23 kB 00:00 2023-01-24T19:13:27.301 INFO:teuthology.orchestra.run.smithi187.stdout:(32/32): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_ 1.7 MB/s | 3.9 MB 00:02 2023-01-24T19:13:27.304 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:27.304 INFO:teuthology.orchestra.run.smithi187.stdout:Total 11 MB/s | 28 MB 00:02 2023-01-24T19:13:27.383 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:13:27.419 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:13:27.419 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:13:27.456 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-24T19:13:27.456 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:27.457 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:13:27.457 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:27.457 INFO:teuthology.orchestra.run.smithi161.stdout: jq-1.6-6.el8.x86_64 2023-01-24T19:13:27.457 INFO:teuthology.orchestra.run.smithi161.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:27.457 INFO:teuthology.orchestra.run.smithi161.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T19:13:27.458 INFO:teuthology.orchestra.run.smithi161.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-24T19:13:27.458 INFO:teuthology.orchestra.run.smithi161.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T19:13:27.458 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:27.458 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:13:27.686 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph 2023-01-24T19:13:27.781 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:13:27.785 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:13:28.424 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:15 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:29.013 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:13:29.285 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-24T19:13:29.538 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-24T19:13:29.573 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:13:29.576 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2023-01-24T19:13:29.576 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:13:29.576 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2023-01-24T19:13:29.577 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:13:29.577 INFO:teuthology.orchestra.run.smithi161.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T19:13:29.577 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:13:29.577 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T19:13:29.578 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T19:13:29.578 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T19:13:29.578 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T19:13:29.578 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T19:13:29.578 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T19:13:29.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T19:13:29.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-24T19:13:29.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T19:13:29.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T19:13:29.579 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-24T19:13:29.580 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T19:13:29.580 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T19:13:29.580 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T19:13:29.580 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-24T19:13:29.580 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-24T19:13:29.581 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T19:13:29.581 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T19:13:29.581 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T19:13:29.581 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-24T19:13:29.581 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-24T19:13:29.581 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-24T19:13:29.582 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T19:13:29.582 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T19:13:29.582 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T19:13:29.582 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T19:13:29.582 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T19:13:29.583 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T19:13:29.583 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T19:13:29.583 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-24T19:13:29.583 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T19:13:29.583 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:29.584 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:13:29.584 INFO:teuthology.orchestra.run.smithi161.stdout:====================================================================================== 2023-01-24T19:13:29.584 INFO:teuthology.orchestra.run.smithi161.stdout:Install 32 Packages 2023-01-24T19:13:29.584 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:29.585 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 28 M 2023-01-24T19:13:29.585 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 110 M 2023-01-24T19:13:29.585 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:13:29.801 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-24T19:13:29.848 INFO:teuthology.orchestra.run.smithi161.stdout:(1/32): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 24 kB/s | 6.4 kB 00:00 2023-01-24T19:13:30.103 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-24T19:13:30.306 INFO:teuthology.orchestra.run.smithi161.stdout:(2/32): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 3.2 MB/s | 2.3 MB 00:00 2023-01-24T19:13:30.682 INFO:teuthology.orchestra.run.smithi161.stdout:(3/32): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 4.7 MB/s | 3.9 MB 00:00 2023-01-24T19:13:30.757 INFO:teuthology.orchestra.run.smithi161.stdout:(4/32): ceph-mgr-modules-core-17.2.5-727.g49f8f 3.0 MB/s | 233 kB 00:00 2023-01-24T19:13:30.816 INFO:teuthology.orchestra.run.smithi161.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-24T19:13:30.864 INFO:teuthology.orchestra.run.smithi161.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 799 kB/s | 39 kB 00:00 2023-01-24T19:13:30.900 INFO:teuthology.orchestra.run.smithi161.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 2.9 MB/s | 103 kB 00:00 2023-01-24T19:13:30.901 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-24T19:13:30.925 INFO:teuthology.orchestra.run.smithi161.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-01-24T19:13:30.958 INFO:teuthology.orchestra.run.smithi161.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-01-24T19:13:30.984 INFO:teuthology.orchestra.run.smithi161.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2023-01-24T19:13:31.026 INFO:teuthology.orchestra.run.smithi161.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 13 MB/s | 559 kB 00:00 2023-01-24T19:13:31.051 INFO:teuthology.orchestra.run.smithi161.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-24T19:13:31.091 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-24T19:13:31.260 INFO:teuthology.orchestra.run.smithi161.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 209 kB/s | 44 kB 00:00 2023-01-24T19:13:31.299 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-24T19:13:31.452 INFO:teuthology.orchestra.run.smithi161.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 966 kB/s | 185 kB 00:00 2023-01-24T19:13:31.477 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-24T19:13:31.577 INFO:teuthology.orchestra.run.smithi161.stdout:(15/32): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_ 801 kB/s | 1.6 MB 00:01 2023-01-24T19:13:31.602 INFO:teuthology.orchestra.run.smithi161.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.1 MB/s | 173 kB 00:00 2023-01-24T19:13:31.669 INFO:teuthology.orchestra.run.smithi161.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2023-01-24T19:13:31.695 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-24T19:13:31.911 INFO:teuthology.orchestra.run.smithi161.stdout:(18/32): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 10 MB/s | 17 MB 00:01 2023-01-24T19:13:31.928 INFO:teuthology.orchestra.run.smithi161.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 68 kB/s | 18 kB 00:00 2023-01-24T19:13:31.957 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-24T19:13:32.020 INFO:teuthology.orchestra.run.smithi161.stdout:(20/32): python3-cherrypy-18.4.0-1.el8.noarch.r 868 kB/s | 384 kB 00:00 2023-01-24T19:13:32.037 INFO:teuthology.orchestra.run.smithi161.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 542 kB/s | 59 kB 00:00 2023-01-24T19:13:32.104 INFO:teuthology.orchestra.run.smithi161.stdout:(22/32): python3-logutils-0.3.5-11.el8.noarch.r 254 kB/s | 49 kB 00:00 2023-01-24T19:13:32.237 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-24T19:13:32.317 INFO:teuthology.orchestra.run.smithi161.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 56 kB/s | 16 kB 00:00 2023-01-24T19:13:32.485 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-24T19:13:32.596 INFO:teuthology.orchestra.run.smithi161.stdout:(24/32): python3-simplegeneric-0.8.1-17.el8.noa 38 kB/s | 19 kB 00:00 2023-01-24T19:13:32.739 INFO:teuthology.orchestra.run.smithi161.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 57 kB/s | 24 kB 00:00 2023-01-24T19:13:32.764 INFO:teuthology.orchestra.run.smithi161.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 175 kB/s | 29 kB 00:00 2023-01-24T19:13:32.789 INFO:teuthology.orchestra.run.smithi161.stdout:(27/32): python3-pecan-1.3.2-9.el8.noarch.rpm 376 kB/s | 283 kB 00:00 2023-01-24T19:13:32.840 INFO:teuthology.orchestra.run.smithi161.stdout:(28/32): python3-trustme-0.6.0-4.el8.noarch.rpm 274 kB/s | 27 kB 00:00 2023-01-24T19:13:32.905 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-24T19:13:32.965 INFO:teuthology.orchestra.run.smithi161.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 683 kB/s | 85 kB 00:00 2023-01-24T19:13:32.990 INFO:teuthology.orchestra.run.smithi161.stdout:(30/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.0 MB/s | 241 kB 00:00 2023-01-24T19:13:33.024 INFO:teuthology.orchestra.run.smithi161.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.0 MB/s | 251 kB 00:00 2023-01-24T19:13:33.049 INFO:teuthology.orchestra.run.smithi161.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 274 kB/s | 23 kB 00:00 2023-01-24T19:13:33.052 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:33.052 INFO:teuthology.orchestra.run.smithi161.stdout:Total 8.1 MB/s | 28 MB 00:03 2023-01-24T19:13:33.138 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:13:33.165 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-24T19:13:33.175 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:13:33.175 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:13:33.370 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-24T19:13:33.551 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:13:33.555 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:13:33.636 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-24T19:13:33.863 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-24T19:13:34.187 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-24T19:13:34.434 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-24T19:13:34.647 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-24T19:13:34.829 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:13:34.892 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-24T19:13:35.108 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-24T19:13:35.175 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-24T19:13:35.384 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-24T19:13:35.424 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-24T19:13:35.647 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-24T19:13:35.657 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-24T19:13:35.983 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-24T19:13:35.986 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-24T19:13:36.206 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-24T19:13:36.340 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-24T19:13:36.386 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-24T19:13:36.612 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-24T19:13:36.753 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 27/32 2023-01-24T19:13:36.833 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-24T19:13:36.996 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 28/32 2023-01-24T19:13:37.065 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-24T19:13:37.344 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-24T19:13:37.615 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-24T19:13:37.874 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-24T19:13:38.302 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-24T19:13:38.478 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 28/32 2023-01-24T19:13:38.588 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-24T19:13:38.668 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 29/32 2023-01-24T19:13:38.834 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-24T19:13:39.064 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 29/32 2023-01-24T19:13:39.109 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-24T19:13:39.270 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 30/32 2023-01-24T19:13:39.385 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-24T19:13:39.485 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 30/32 2023-01-24T19:13:39.670 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 31/32 2023-01-24T19:13:39.686 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-24T19:13:39.688 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 31/32 2023-01-24T19:13:39.859 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 32/32 2023-01-24T19:13:39.907 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-24T19:13:40.153 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-24T19:13:40.298 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 32/32 2023-01-24T19:13:40.299 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/32 2023-01-24T19:13:40.299 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/32 2023-01-24T19:13:40.300 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/32 2023-01-24T19:13:40.300 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/32 2023-01-24T19:13:40.300 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/32 2023-01-24T19:13:40.300 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/32 2023-01-24T19:13:40.301 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-24T19:13:40.301 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-24T19:13:40.301 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-24T19:13:40.302 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-24T19:13:40.302 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-24T19:13:40.302 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-24T19:13:40.302 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-24T19:13:40.303 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-24T19:13:40.303 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-24T19:13:40.303 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-24T19:13:40.304 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-24T19:13:40.304 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-24T19:13:40.304 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-24T19:13:40.304 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-24T19:13:40.305 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-24T19:13:40.305 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-24T19:13:40.305 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-24T19:13:40.305 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-24T19:13:40.305 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-24T19:13:40.306 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-24T19:13:40.306 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-24T19:13:40.306 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-24T19:13:40.306 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-24T19:13:40.306 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-24T19:13:40.307 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-24T19:13:40.406 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-24T19:13:40.675 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-24T19:13:40.949 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-24T19:13:41.195 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-24T19:13:41.312 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-24T19:13:41.312 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:41.313 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:13:41.313 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:41.313 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:41.313 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:41.313 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:13:41.314 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:41.314 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:41.314 INFO:teuthology.orchestra.run.smithi187.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T19:13:41.314 INFO:teuthology.orchestra.run.smithi187.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T19:13:41.314 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-24T19:13:41.314 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T19:13:41.315 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T19:13:41.315 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-24T19:13:41.315 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T19:13:41.315 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T19:13:41.315 INFO:teuthology.orchestra.run.smithi187.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T19:13:41.316 INFO:teuthology.orchestra.run.smithi187.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T19:13:41.316 INFO:teuthology.orchestra.run.smithi187.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T19:13:41.316 INFO:teuthology.orchestra.run.smithi187.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T19:13:41.316 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T19:13:41.316 INFO:teuthology.orchestra.run.smithi187.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T19:13:41.317 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-24T19:13:41.317 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-24T19:13:41.317 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T19:13:41.317 INFO:teuthology.orchestra.run.smithi187.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T19:13:41.318 INFO:teuthology.orchestra.run.smithi187.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T19:13:41.318 INFO:teuthology.orchestra.run.smithi187.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T19:13:41.318 INFO:teuthology.orchestra.run.smithi187.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T19:13:41.318 INFO:teuthology.orchestra.run.smithi187.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T19:13:41.319 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T19:13:41.319 INFO:teuthology.orchestra.run.smithi187.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T19:13:41.319 INFO:teuthology.orchestra.run.smithi187.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T19:13:41.319 INFO:teuthology.orchestra.run.smithi187.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T19:13:41.320 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:41.320 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:13:41.448 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-base 2023-01-24T19:13:41.608 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-24T19:13:42.007 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-24T19:13:42.169 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:29 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:42.431 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 27/32 2023-01-24T19:13:42.665 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 28/32 2023-01-24T19:13:43.213 INFO:teuthology.orchestra.run.smithi187.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:13:43.311 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:13:43.312 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:13:43.313 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:13:43.404 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install cephadm 2023-01-24T19:13:44.119 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:31 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:44.166 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 28/32 2023-01-24T19:13:44.358 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 29/32 2023-01-24T19:13:44.753 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 29/32 2023-01-24T19:13:44.934 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 30/32 2023-01-24T19:13:45.151 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 30/32 2023-01-24T19:13:45.255 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:13:45.256 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:45.256 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:13:45.257 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:45.257 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:13:45.257 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T19:13:45.257 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:45.258 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:13:45.258 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:45.258 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:13:45.258 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:45.258 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 78 k 2023-01-24T19:13:45.258 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 355 k 2023-01-24T19:13:45.259 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:13:45.309 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 31/32 2023-01-24T19:13:45.327 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 31/32 2023-01-24T19:13:45.490 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 32/32 2023-01-24T19:13:45.589 INFO:teuthology.orchestra.run.smithi187.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 235 kB/s | 78 kB 00:00 2023-01-24T19:13:45.590 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:45.590 INFO:teuthology.orchestra.run.smithi187.stdout:Total 235 kB/s | 78 kB 00:00 2023-01-24T19:13:45.591 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:13:45.598 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:13:45.598 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:13:45.600 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:13:45.603 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:13:45.654 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:13:45.935 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 32/32 2023-01-24T19:13:45.936 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/32 2023-01-24T19:13:45.936 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/32 2023-01-24T19:13:45.936 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/32 2023-01-24T19:13:45.936 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/32 2023-01-24T19:13:45.936 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/32 2023-01-24T19:13:45.937 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/32 2023-01-24T19:13:45.937 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-24T19:13:45.937 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-24T19:13:45.937 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-24T19:13:45.937 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-24T19:13:45.938 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-24T19:13:45.938 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-24T19:13:45.938 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-24T19:13:45.938 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-24T19:13:45.938 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-24T19:13:45.939 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-24T19:13:45.939 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-24T19:13:45.939 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-24T19:13:45.939 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-24T19:13:45.939 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-24T19:13:45.940 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-24T19:13:45.940 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-24T19:13:45.940 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-24T19:13:45.940 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-24T19:13:45.940 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-24T19:13:45.941 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-24T19:13:45.941 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-24T19:13:45.941 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-24T19:13:45.941 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-24T19:13:45.941 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-24T19:13:45.941 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-24T19:13:46.894 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-24T19:13:46.895 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:46.895 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:13:46.895 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:46.895 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:46.895 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:46.896 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:13:46.896 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:46.896 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:46.896 INFO:teuthology.orchestra.run.smithi161.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T19:13:46.896 INFO:teuthology.orchestra.run.smithi161.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T19:13:46.897 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-24T19:13:46.897 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T19:13:46.897 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T19:13:46.897 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-24T19:13:46.897 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T19:13:46.898 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T19:13:46.898 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T19:13:46.898 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T19:13:46.898 INFO:teuthology.orchestra.run.smithi161.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T19:13:46.898 INFO:teuthology.orchestra.run.smithi161.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T19:13:46.899 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T19:13:46.899 INFO:teuthology.orchestra.run.smithi161.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T19:13:46.899 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-24T19:13:46.899 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-24T19:13:46.899 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T19:13:46.900 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T19:13:46.900 INFO:teuthology.orchestra.run.smithi161.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T19:13:46.900 INFO:teuthology.orchestra.run.smithi161.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T19:13:46.900 INFO:teuthology.orchestra.run.smithi161.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T19:13:46.901 INFO:teuthology.orchestra.run.smithi161.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T19:13:46.901 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T19:13:46.901 INFO:teuthology.orchestra.run.smithi161.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T19:13:46.901 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T19:13:46.901 INFO:teuthology.orchestra.run.smithi161.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T19:13:46.901 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:46.902 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:13:47.147 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-base 2023-01-24T19:13:47.353 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:47.566 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:47.803 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:47.879 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:34 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:48.289 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:48.290 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:48.290 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:13:48.291 INFO:teuthology.orchestra.run.smithi187.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:13:48.291 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:48.291 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:13:48.604 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T19:13:48.925 INFO:teuthology.orchestra.run.smithi161.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:13:49.026 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:13:49.028 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:13:49.028 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:13:49.119 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install cephadm 2023-01-24T19:13:49.334 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:36 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:49.835 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:36 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:50.473 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:13:50.474 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:50.474 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repo Size 2023-01-24T19:13:50.474 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:50.474 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:13:50.475 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T19:13:50.475 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:50.475 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:13:50.475 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:50.475 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:13:50.476 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:50.476 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 162 k 2023-01-24T19:13:50.476 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 490 k 2023-01-24T19:13:50.476 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:13:50.946 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 343 kB/s | 162 kB 00:00 2023-01-24T19:13:50.947 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:50.947 INFO:teuthology.orchestra.run.smithi187.stdout:Total 342 kB/s | 162 kB 00:00 2023-01-24T19:13:50.948 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:13:50.958 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:13:50.958 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:13:50.973 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:13:50.974 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:50.974 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:13:50.975 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:50.975 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:13:50.975 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T19:13:50.975 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:50.975 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:13:50.976 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:50.976 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:13:50.976 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:50.976 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 78 k 2023-01-24T19:13:50.976 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 355 k 2023-01-24T19:13:50.977 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:13:51.006 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:13:51.010 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:13:51.120 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:13:51.202 INFO:teuthology.orchestra.run.smithi161.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 345 kB/s | 78 kB 00:00 2023-01-24T19:13:51.203 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:51.203 INFO:teuthology.orchestra.run.smithi161.stdout:Total 343 kB/s | 78 kB 00:00 2023-01-24T19:13:51.203 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:13:51.210 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:13:51.210 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:13:51.212 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:13:51.216 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:13:51.267 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:13:52.227 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:13:52.888 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:53.064 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:53.299 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:53.410 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:13:53.928 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:13:53.929 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:53.929 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:13:53.929 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:53.929 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:53.930 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:13:54.124 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-mgr 2023-01-24T19:13:54.465 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:13:54.465 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:54.466 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:13:54.466 INFO:teuthology.orchestra.run.smithi161.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:13:54.466 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:54.466 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:13:54.872 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:41 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:55.657 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T19:13:55.913 INFO:teuthology.orchestra.run.smithi187.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:13:56.012 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:13:56.013 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:13:56.014 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:13:56.094 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T19:13:56.397 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:43 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:56.818 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:43 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:13:57.545 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:13:57.546 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:57.546 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2023-01-24T19:13:57.547 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:57.547 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:13:57.547 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T19:13:57.547 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:57.547 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:13:57.548 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:13:57.548 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:13:57.548 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:57.548 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 162 k 2023-01-24T19:13:57.548 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 490 k 2023-01-24T19:13:57.549 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:13:57.965 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:13:57.966 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:57.966 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:13:57.967 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:57.967 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:13:57.967 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T19:13:57.967 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:13:57.967 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T19:13:57.968 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T19:13:57.968 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-24T19:13:57.968 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T19:13:57.968 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T19:13:57.968 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:57.969 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:13:57.969 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:13:57.969 INFO:teuthology.orchestra.run.smithi187.stdout:Install 6 Packages 2023-01-24T19:13:57.969 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:13:57.970 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 2.0 M 2023-01-24T19:13:57.970 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 7.8 M 2023-01-24T19:13:57.970 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:13:57.975 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 378 kB/s | 162 kB 00:00 2023-01-24T19:13:57.975 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:57.976 INFO:teuthology.orchestra.run.smithi161.stdout:Total 377 kB/s | 162 kB 00:00 2023-01-24T19:13:57.976 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:13:57.987 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:13:57.987 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:13:58.037 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:13:58.041 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:13:58.157 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:13:58.283 INFO:teuthology.orchestra.run.smithi187.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 73 kB/s | 23 kB 00:00 2023-01-24T19:13:58.309 INFO:teuthology.orchestra.run.smithi187.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 43 kB/s | 14 kB 00:00 2023-01-24T19:13:58.340 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:13:58.380 INFO:teuthology.orchestra.run.smithi187.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 441 kB/s | 43 kB 00:00 2023-01-24T19:13:58.734 INFO:teuthology.orchestra.run.smithi187.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 79 kB/s | 34 kB 00:00 2023-01-24T19:13:58.825 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:13:59.296 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T19:13:59.296 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:59.296 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:13:59.297 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:13:59.297 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:13:59.297 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:13:59.418 INFO:teuthology.orchestra.run.smithi187.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 189 kB/s | 196 kB 00:01 2023-01-24T19:13:59.468 INFO:teuthology.orchestra.run.smithi187.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 1.1 MB/s | 1.6 MB 00:01 2023-01-24T19:13:59.471 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:13:59.471 INFO:teuthology.orchestra.run.smithi187.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2023-01-24T19:13:59.481 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:13:59.490 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:13:59.490 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:13:59.505 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr 2023-01-24T19:13:59.573 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:13:59.577 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:13:59.861 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:00.105 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-24T19:14:00.244 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:47 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:00.375 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-24T19:14:00.602 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-24T19:14:00.713 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T19:14:01.086 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T19:14:01.283 INFO:teuthology.orchestra.run.smithi161.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:01.346 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:14:01.383 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:01.384 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:14:01.384 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:01.488 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T19:14:01.648 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:14:01.648 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T19:14:01.648 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T19:14:01.649 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T19:14:01.649 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-24T19:14:01.649 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-24T19:14:02.125 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-24T19:14:02.125 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:02.125 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:02.126 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:02.126 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:02.126 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:02.126 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T19:14:02.126 INFO:teuthology.orchestra.run.smithi187.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T19:14:02.127 INFO:teuthology.orchestra.run.smithi187.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T19:14:02.127 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:02.127 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:02.214 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:49 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:02.230 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T19:14:02.950 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:01:49 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:03.366 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:03.367 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:03.367 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:14:03.367 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:03.368 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:03.368 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T19:14:03.368 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:14:03.368 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T19:14:03.368 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T19:14:03.369 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-24T19:14:03.369 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T19:14:03.369 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T19:14:03.369 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:03.369 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:03.370 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:03.370 INFO:teuthology.orchestra.run.smithi161.stdout:Install 6 Packages 2023-01-24T19:14:03.370 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:03.371 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 2.0 M 2023-01-24T19:14:03.371 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 7.8 M 2023-01-24T19:14:03.371 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:03.680 INFO:teuthology.orchestra.run.smithi161.stdout:(1/6): ceph-grafana-dashboards-17.2.5-727.g49f8 74 kB/s | 23 kB 00:00 2023-01-24T19:14:03.696 INFO:teuthology.orchestra.run.smithi161.stdout:(2/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 44 kB/s | 14 kB 00:00 2023-01-24T19:14:03.813 INFO:teuthology.orchestra.run.smithi161.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 319 kB/s | 43 kB 00:00 2023-01-24T19:14:03.906 INFO:teuthology.orchestra.run.smithi161.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 161 kB/s | 34 kB 00:00 2023-01-24T19:14:04.091 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:04.092 INFO:teuthology.orchestra.run.smithi187.stdout:===================================================================================================== 2023-01-24T19:14:04.092 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:14:04.092 INFO:teuthology.orchestra.run.smithi187.stdout:===================================================================================================== 2023-01-24T19:14:04.093 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:14:04.093 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T19:14:04.093 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:14:04.093 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-24T19:14:04.093 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-24T19:14:04.094 INFO:teuthology.orchestra.run.smithi187.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-24T19:14:04.094 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-24T19:14:04.094 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-24T19:14:04.094 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-24T19:14:04.094 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-24T19:14:04.094 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:04.095 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:14:04.095 INFO:teuthology.orchestra.run.smithi187.stdout:===================================================================================================== 2023-01-24T19:14:04.095 INFO:teuthology.orchestra.run.smithi187.stdout:Install 8 Packages 2023-01-24T19:14:04.095 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:04.096 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 35 M 2023-01-24T19:14:04.096 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 210 M 2023-01-24T19:14:04.096 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:14:04.181 INFO:teuthology.orchestra.run.smithi161.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 535 kB/s | 196 kB 00:00 2023-01-24T19:14:04.435 INFO:teuthology.orchestra.run.smithi187.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-01-24T19:14:04.569 INFO:teuthology.orchestra.run.smithi187.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 28 MB/s | 3.7 MB 00:00 2023-01-24T19:14:04.611 INFO:teuthology.orchestra.run.smithi187.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.3 MB/s | 225 kB 00:00 2023-01-24T19:14:04.736 INFO:teuthology.orchestra.run.smithi187.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 7.6 MB/s | 4.9 MB 00:00 2023-01-24T19:14:04.803 INFO:teuthology.orchestra.run.smithi187.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 9.5 MB/s | 645 kB 00:00 2023-01-24T19:14:04.845 INFO:teuthology.orchestra.run.smithi187.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 4.0 MB/s | 172 kB 00:00 2023-01-24T19:14:04.987 INFO:teuthology.orchestra.run.smithi187.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 36 MB/s | 14 MB 00:00 2023-01-24T19:14:05.053 INFO:teuthology.orchestra.run.smithi161.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 1.0 MB/s | 1.6 MB 00:01 2023-01-24T19:14:05.056 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:05.056 INFO:teuthology.orchestra.run.smithi161.stdout:Total 1.2 MB/s | 2.0 MB 00:01 2023-01-24T19:14:05.070 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:05.078 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:05.078 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:05.166 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:05.170 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:05.481 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:05.686 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-24T19:14:06.000 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-24T19:14:06.249 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-24T19:14:06.370 INFO:teuthology.orchestra.run.smithi187.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 3.3 MB/s | 7.4 MB 00:02 2023-01-24T19:14:06.373 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:06.373 INFO:teuthology.orchestra.run.smithi187.stdout:Total 16 MB/s | 35 MB 00:02 2023-01-24T19:14:06.403 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T19:14:06.607 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:14:06.625 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:14:06.625 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:14:06.758 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T19:14:06.979 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:14:07.017 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:14:07.021 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:14:07.284 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T19:14:07.284 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T19:14:07.284 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T19:14:07.285 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T19:14:07.285 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-24T19:14:07.285 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-24T19:14:07.606 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:07.764 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-24T19:14:07.794 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-24T19:14:07.795 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:07.795 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:07.795 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:07.796 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:07.796 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:07.797 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T19:14:07.797 INFO:teuthology.orchestra.run.smithi161.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T19:14:07.797 INFO:teuthology.orchestra.run.smithi161.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T19:14:07.797 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:07.798 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:07.895 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-24T19:14:07.907 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T19:14:08.089 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-24T19:14:08.625 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:01:55 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:08.686 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-24T19:14:08.889 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-24T19:14:09.523 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-24T19:14:09.771 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:09.772 INFO:teuthology.orchestra.run.smithi161.stdout:===================================================================================================== 2023-01-24T19:14:09.773 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:14:09.773 INFO:teuthology.orchestra.run.smithi161.stdout:===================================================================================================== 2023-01-24T19:14:09.773 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:09.774 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T19:14:09.774 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:14:09.774 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-24T19:14:09.775 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-24T19:14:09.775 INFO:teuthology.orchestra.run.smithi161.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-24T19:14:09.775 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-24T19:14:09.776 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-24T19:14:09.776 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-24T19:14:09.776 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-24T19:14:09.776 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:09.777 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:09.777 INFO:teuthology.orchestra.run.smithi161.stdout:===================================================================================================== 2023-01-24T19:14:09.777 INFO:teuthology.orchestra.run.smithi161.stdout:Install 8 Packages 2023-01-24T19:14:09.777 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:09.779 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 35 M 2023-01-24T19:14:09.779 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 210 M 2023-01-24T19:14:09.779 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:09.987 INFO:teuthology.orchestra.run.smithi161.stdout:(1/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 23 MB/s | 4.9 MB 00:00 2023-01-24T19:14:10.126 INFO:teuthology.orchestra.run.smithi161.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 27 MB/s | 3.7 MB 00:00 2023-01-24T19:14:10.163 INFO:teuthology.orchestra.run.smithi161.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.9 MB/s | 225 kB 00:00 2023-01-24T19:14:10.198 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-24T19:14:10.463 INFO:teuthology.orchestra.run.smithi161.stdout:(4/8): python3-scipy-1.0.0-21.module_el8.5.0+77 46 MB/s | 14 MB 00:00 2023-01-24T19:14:10.497 INFO:teuthology.orchestra.run.smithi161.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 19 MB/s | 645 kB 00:00 2023-01-24T19:14:10.522 INFO:teuthology.orchestra.run.smithi161.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 6.7 MB/s | 172 kB 00:00 2023-01-24T19:14:10.631 INFO:teuthology.orchestra.run.smithi161.stdout:(7/8): openblas-0.3.15-4.el8.x86_64.rpm 5.5 MB/s | 4.7 MB 00:00 2023-01-24T19:14:10.895 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-24T19:14:11.239 INFO:teuthology.orchestra.run.smithi161.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 5.1 MB/s | 7.4 MB 00:01 2023-01-24T19:14:11.242 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:11.242 INFO:teuthology.orchestra.run.smithi161.stdout:Total 24 MB/s | 35 MB 00:01 2023-01-24T19:14:11.250 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-24T19:14:11.474 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:11.480 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-24T19:14:11.492 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:11.492 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:11.897 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:11.900 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:12.511 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:12.712 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-24T19:14:12.850 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-24T19:14:13.037 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-24T19:14:13.294 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-24T19:14:13.644 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-24T19:14:13.837 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-24T19:14:14.456 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-24T19:14:14.560 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-24T19:14:14.671 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-24T19:14:14.821 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:14:15.160 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:14:15.161 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T19:14:15.161 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-24T19:14:15.161 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-24T19:14:15.161 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-24T19:14:15.162 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-24T19:14:15.162 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-24T19:14:15.162 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-24T19:14:15.362 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-24T19:14:15.668 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-24T19:14:15.668 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:15.669 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:15.669 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:15.669 INFO:teuthology.orchestra.run.smithi187.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-24T19:14:15.669 INFO:teuthology.orchestra.run.smithi187.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-24T19:14:15.669 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-24T19:14:15.670 INFO:teuthology.orchestra.run.smithi187.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-24T19:14:15.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-24T19:14:15.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-24T19:14:15.670 INFO:teuthology.orchestra.run.smithi187.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-24T19:14:15.671 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:15.671 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:15.750 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-24T19:14:15.772 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-mgr-rook 2023-01-24T19:14:16.003 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-24T19:14:16.507 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:03 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:17.645 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:17.646 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2023-01-24T19:14:17.646 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:14:17.647 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2023-01-24T19:14:17.647 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:14:17.647 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T19:14:17.647 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:14:17.648 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T19:14:17.648 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T19:14:17.648 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T19:14:17.648 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-24T19:14:17.649 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-24T19:14:17.649 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T19:14:17.650 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-24T19:14:17.650 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-24T19:14:17.650 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-24T19:14:17.651 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-24T19:14:17.651 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T19:14:17.651 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T19:14:17.652 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:17.652 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:14:17.652 INFO:teuthology.orchestra.run.smithi187.stdout:======================================================================================= 2023-01-24T19:14:17.653 INFO:teuthology.orchestra.run.smithi187.stdout:Install 13 Packages 2023-01-24T19:14:17.653 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:17.654 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 1.9 M 2023-01-24T19:14:17.655 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 25 M 2023-01-24T19:14:17.655 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:14:17.726 INFO:teuthology.orchestra.run.smithi187.stdout:(1/13): python3-jsonpointer-1.10-11.el8.noarch. 262 kB/s | 20 kB 00:00 2023-01-24T19:14:17.752 INFO:teuthology.orchestra.run.smithi187.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 261 kB/s | 27 kB 00:00 2023-01-24T19:14:17.777 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-24T19:14:17.777 INFO:teuthology.orchestra.run.smithi187.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-24T19:14:17.794 INFO:teuthology.orchestra.run.smithi187.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.6 MB/s | 110 kB 00:00 2023-01-24T19:14:17.819 INFO:teuthology.orchestra.run.smithi187.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.6 MB/s | 155 kB 00:00 2023-01-24T19:14:17.845 INFO:teuthology.orchestra.run.smithi187.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 846 kB/s | 43 kB 00:00 2023-01-24T19:14:17.870 INFO:teuthology.orchestra.run.smithi187.stdout:(7/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 213 kB/s | 47 kB 00:00 2023-01-24T19:14:17.962 INFO:teuthology.orchestra.run.smithi187.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 231 kB/s | 33 kB 00:00 2023-01-24T19:14:17.987 INFO:teuthology.orchestra.run.smithi187.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 114 kB/s | 16 kB 00:00 2023-01-24T19:14:18.004 INFO:teuthology.orchestra.run.smithi187.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 607 kB/s | 81 kB 00:00 2023-01-24T19:14:18.055 INFO:teuthology.orchestra.run.smithi187.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 1.2 MB/s | 61 kB 00:00 2023-01-24T19:14:18.080 INFO:teuthology.orchestra.run.smithi187.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 669 kB/s | 61 kB 00:00 2023-01-24T19:14:18.388 INFO:teuthology.orchestra.run.smithi187.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.7 MB/s | 1.2 MB 00:00 2023-01-24T19:14:18.390 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:18.390 INFO:teuthology.orchestra.run.smithi187.stdout:Total 2.6 MB/s | 1.9 MB 00:00 2023-01-24T19:14:18.442 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:14:18.457 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:14:18.457 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:14:18.554 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:14:18.558 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:14:19.071 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-24T19:14:19.072 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:19.295 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:14:19.357 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T19:14:19.589 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-24T19:14:19.621 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T19:14:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T19:14:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-24T19:14:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-24T19:14:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-24T19:14:19.622 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-24T19:14:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-24T19:14:19.623 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-24T19:14:19.856 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-24T19:14:20.094 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-24T19:14:20.135 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-24T19:14:20.135 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:20.135 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:20.135 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:20.136 INFO:teuthology.orchestra.run.smithi161.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-24T19:14:20.136 INFO:teuthology.orchestra.run.smithi161.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-24T19:14:20.136 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-24T19:14:20.136 INFO:teuthology.orchestra.run.smithi161.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-24T19:14:20.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-24T19:14:20.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-24T19:14:20.137 INFO:teuthology.orchestra.run.smithi161.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-24T19:14:20.137 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:20.137 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:20.289 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-rook 2023-01-24T19:14:20.310 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-24T19:14:20.936 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-24T19:14:21.012 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:08 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:21.253 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-24T19:14:21.534 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T19:14:22.157 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:22.158 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2023-01-24T19:14:22.158 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:14:22.159 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2023-01-24T19:14:22.159 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:22.159 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T19:14:22.159 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:14:22.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T19:14:22.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T19:14:22.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T19:14:22.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-24T19:14:22.160 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-24T19:14:22.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T19:14:22.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-24T19:14:22.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-24T19:14:22.161 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-24T19:14:22.162 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-24T19:14:22.162 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T19:14:22.162 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T19:14:22.162 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:22.162 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:22.163 INFO:teuthology.orchestra.run.smithi161.stdout:======================================================================================= 2023-01-24T19:14:22.163 INFO:teuthology.orchestra.run.smithi161.stdout:Install 13 Packages 2023-01-24T19:14:22.163 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:22.164 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 1.9 M 2023-01-24T19:14:22.164 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 25 M 2023-01-24T19:14:22.164 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:22.256 INFO:teuthology.orchestra.run.smithi161.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 282 kB/s | 27 kB 00:00 2023-01-24T19:14:22.281 INFO:teuthology.orchestra.run.smithi161.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 168 kB/s | 20 kB 00:00 2023-01-24T19:14:22.307 INFO:teuthology.orchestra.run.smithi161.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-24T19:14:22.332 INFO:teuthology.orchestra.run.smithi161.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-24T19:14:22.357 INFO:teuthology.orchestra.run.smithi161.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-24T19:14:22.374 INFO:teuthology.orchestra.run.smithi161.stdout:(6/13): python3-requests-oauthlib-1.0.0-1.el8.n 1.0 MB/s | 43 kB 00:00 2023-01-24T19:14:22.433 INFO:teuthology.orchestra.run.smithi161.stdout:(7/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 173 kB/s | 47 kB 00:00 2023-01-24T19:14:22.508 INFO:teuthology.orchestra.run.smithi161.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 122 kB/s | 16 kB 00:00 2023-01-24T19:14:22.604 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T19:14:22.633 INFO:teuthology.orchestra.run.smithi161.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 119 kB/s | 33 kB 00:00 2023-01-24T19:14:22.742 INFO:teuthology.orchestra.run.smithi161.stdout:(10/13): python3-rsa-4.8-1.el8.noarch.rpm 568 kB/s | 61 kB 00:00 2023-01-24T19:14:22.910 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-24T19:14:22.985 INFO:teuthology.orchestra.run.smithi161.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 252 kB/s | 61 kB 00:00 2023-01-24T19:14:23.169 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-24T19:14:23.409 INFO:teuthology.orchestra.run.smithi161.stdout:(12/13): python3-kubernetes-11.0.0-6.el8.noarch 1.3 MB/s | 1.2 MB 00:00 2023-01-24T19:14:23.412 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-24T19:14:23.434 INFO:teuthology.orchestra.run.smithi161.stdout:(13/13): python3-google-auth-1.1.1-10.el8.noarc 81 kB/s | 81 kB 00:01 2023-01-24T19:14:23.437 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:23.437 INFO:teuthology.orchestra.run.smithi161.stdout:Total 1.5 MB/s | 1.9 MB 00:01 2023-01-24T19:14:23.496 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:23.512 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:23.512 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:23.586 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:14:23.610 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:23.614 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:23.900 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:14:23.900 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T19:14:23.901 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-24T19:14:23.901 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-24T19:14:23.901 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-24T19:14:23.901 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-24T19:14:23.902 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T19:14:23.902 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T19:14:23.902 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-24T19:14:23.902 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T19:14:23.903 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-24T19:14:23.903 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-24T19:14:23.903 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-24T19:14:24.131 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:24.446 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T19:14:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-24T19:14:24.482 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:24.483 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T19:14:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T19:14:24.483 INFO:teuthology.orchestra.run.smithi187.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T19:14:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T19:14:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T19:14:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T19:14:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T19:14:24.484 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T19:14:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T19:14:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T19:14:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T19:14:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T19:14:24.485 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:24.486 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:24.613 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T19:14:24.637 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-24T19:14:24.913 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-24T19:14:25.167 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-24T19:14:25.343 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:12 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:25.349 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-24T19:14:25.606 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-24T19:14:25.931 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-24T19:14:26.207 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T19:14:26.479 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:26.480 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:26.480 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:14:26.480 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:26.481 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:14:26.481 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T19:14:26.481 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:14:26.481 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T19:14:26.481 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-24T19:14:26.482 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-24T19:14:26.482 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T19:14:26.482 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:26.482 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:14:26.482 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:26.483 INFO:teuthology.orchestra.run.smithi187.stdout:Install 5 Packages 2023-01-24T19:14:26.483 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:26.483 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 5.9 M 2023-01-24T19:14:26.484 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 26 M 2023-01-24T19:14:26.484 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:14:26.620 INFO:teuthology.orchestra.run.smithi187.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 3.8 MB/s | 538 kB 00:00 2023-01-24T19:14:26.704 INFO:teuthology.orchestra.run.smithi187.stdout:(2/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 519 kB/s | 115 kB 00:00 2023-01-24T19:14:26.880 INFO:teuthology.orchestra.run.smithi187.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-01-24T19:14:26.905 INFO:teuthology.orchestra.run.smithi187.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 301 kB/s | 60 kB 00:00 2023-01-24T19:14:27.005 INFO:teuthology.orchestra.run.smithi187.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.2 MB/s | 476 kB 00:00 2023-01-24T19:14:27.007 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:27.007 INFO:teuthology.orchestra.run.smithi187.stdout:Total 11 MB/s | 5.9 MB 00:00 2023-01-24T19:14:27.058 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:14:27.069 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:14:27.069 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:14:27.212 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:14:27.215 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:14:27.325 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T19:14:27.487 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:27.600 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-24T19:14:27.798 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-24T19:14:27.851 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-24T19:14:28.086 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-24T19:14:28.226 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:14:28.562 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T19:14:28.562 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T19:14:28.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-24T19:14:28.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-24T19:14:28.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-24T19:14:28.563 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-24T19:14:28.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T19:14:28.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T19:14:28.564 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-24T19:14:28.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T19:14:28.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-24T19:14:28.565 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-24T19:14:28.566 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-24T19:14:28.782 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T19:14:29.149 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-24T19:14:29.149 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:29.149 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:29.150 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:29.150 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T19:14:29.150 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T19:14:29.150 INFO:teuthology.orchestra.run.smithi161.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T19:14:29.150 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T19:14:29.151 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T19:14:29.151 INFO:teuthology.orchestra.run.smithi161.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T19:14:29.151 INFO:teuthology.orchestra.run.smithi161.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T19:14:29.151 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T19:14:29.151 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T19:14:29.152 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T19:14:29.152 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T19:14:29.152 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T19:14:29.152 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:29.152 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:29.181 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-24T19:14:29.330 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T19:14:29.454 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-24T19:14:29.669 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:14:29.980 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:14:29.980 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T19:14:29.980 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-24T19:14:29.981 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-24T19:14:29.981 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T19:14:30.051 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:17 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:30.489 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-24T19:14:30.489 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:30.489 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:30.489 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:30.489 INFO:teuthology.orchestra.run.smithi187.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T19:14:30.490 INFO:teuthology.orchestra.run.smithi187.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-24T19:14:30.490 INFO:teuthology.orchestra.run.smithi187.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-24T19:14:30.490 INFO:teuthology.orchestra.run.smithi187.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T19:14:30.490 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:30.490 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:30.594 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-fuse 2023-01-24T19:14:31.177 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:31.178 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:31.178 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:14:31.179 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:31.179 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:31.179 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T19:14:31.179 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:14:31.179 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T19:14:31.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-24T19:14:31.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-24T19:14:31.180 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T19:14:31.180 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:31.180 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:31.181 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:31.181 INFO:teuthology.orchestra.run.smithi161.stdout:Install 5 Packages 2023-01-24T19:14:31.181 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:31.182 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 5.9 M 2023-01-24T19:14:31.182 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 26 M 2023-01-24T19:14:31.182 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:31.257 INFO:teuthology.orchestra.run.smithi161.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 6.9 MB/s | 538 kB 00:00 2023-01-24T19:14:31.327 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:18 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:31.374 INFO:teuthology.orchestra.run.smithi161.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 25 MB/s | 4.8 MB 00:00 2023-01-24T19:14:31.600 INFO:teuthology.orchestra.run.smithi161.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 268 kB/s | 60 kB 00:00 2023-01-24T19:14:31.625 INFO:teuthology.orchestra.run.smithi161.stdout:(4/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 259 kB/s | 115 kB 00:00 2023-01-24T19:14:31.700 INFO:teuthology.orchestra.run.smithi161.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.1 MB/s | 476 kB 00:00 2023-01-24T19:14:31.702 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:31.702 INFO:teuthology.orchestra.run.smithi161.stdout:Total 11 MB/s | 5.9 MB 00:00 2023-01-24T19:14:31.754 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:31.765 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:31.765 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:31.900 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:31.904 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:32.217 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:32.463 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:32.464 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:32.464 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:14:32.465 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:32.465 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:14:32.465 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T19:14:32.465 INFO:teuthology.orchestra.run.smithi187.stdout:Installing dependencies: 2023-01-24T19:14:32.465 INFO:teuthology.orchestra.run.smithi187.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-24T19:14:32.466 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:32.466 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:14:32.466 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:32.466 INFO:teuthology.orchestra.run.smithi187.stdout:Install 2 Packages 2023-01-24T19:14:32.466 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:32.467 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 954 k 2023-01-24T19:14:32.467 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 3.0 M 2023-01-24T19:14:32.467 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:14:32.537 INFO:teuthology.orchestra.run.smithi187.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-01-24T19:14:32.573 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-24T19:14:32.754 INFO:teuthology.orchestra.run.smithi187.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 3.0 MB/s | 872 kB 00:00 2023-01-24T19:14:32.755 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:32.755 INFO:teuthology.orchestra.run.smithi187.stdout:Total 3.2 MB/s | 954 kB 00:00 2023-01-24T19:14:32.765 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:14:32.786 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:14:32.786 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:14:32.879 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:14:32.882 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:14:33.083 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:33.360 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-24T19:14:33.540 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T19:14:33.591 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:14:33.946 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-24T19:14:34.008 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:14:34.008 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T19:14:34.261 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-24T19:14:34.492 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:14:34.507 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-24T19:14:34.508 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:34.508 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:34.508 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-24T19:14:34.508 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:34.508 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:34.687 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install ceph-volume 2023-01-24T19:14:34.799 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T19:14:34.799 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T19:14:34.799 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-24T19:14:34.800 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-24T19:14:34.800 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T19:14:35.274 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-24T19:14:35.274 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:35.275 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:35.275 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:35.275 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T19:14:35.275 INFO:teuthology.orchestra.run.smithi161.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-24T19:14:35.275 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-24T19:14:35.276 INFO:teuthology.orchestra.run.smithi161.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T19:14:35.276 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:35.276 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:35.411 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-fuse 2023-01-24T19:14:35.427 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:22 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:36.158 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:23 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:36.567 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:36.568 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:36.569 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repository Size 2023-01-24T19:14:36.569 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:36.569 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:14:36.569 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T19:14:36.570 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:36.570 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:14:36.570 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:36.570 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:14:36.570 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:36.571 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 263 k 2023-01-24T19:14:36.571 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 1.3 M 2023-01-24T19:14:36.571 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:14:36.902 INFO:teuthology.orchestra.run.smithi187.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 789 kB/s | 263 kB 00:00 2023-01-24T19:14:36.903 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:36.903 INFO:teuthology.orchestra.run.smithi187.stdout:Total 786 kB/s | 263 kB 00:00 2023-01-24T19:14:36.904 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:14:36.913 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:14:36.914 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:14:37.012 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:14:37.016 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:14:37.200 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:37.297 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:37.298 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:37.298 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:14:37.298 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:37.299 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:37.299 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T19:14:37.299 INFO:teuthology.orchestra.run.smithi161.stdout:Installing dependencies: 2023-01-24T19:14:37.299 INFO:teuthology.orchestra.run.smithi161.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-24T19:14:37.299 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:37.300 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:37.300 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:37.300 INFO:teuthology.orchestra.run.smithi161.stdout:Install 2 Packages 2023-01-24T19:14:37.300 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:37.301 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 954 k 2023-01-24T19:14:37.301 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 3.0 M 2023-01-24T19:14:37.301 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:37.365 INFO:teuthology.orchestra.run.smithi161.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-01-24T19:14:37.468 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:14:37.682 INFO:teuthology.orchestra.run.smithi161.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 2.2 MB/s | 872 kB 00:00 2023-01-24T19:14:37.683 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:37.684 INFO:teuthology.orchestra.run.smithi161.stdout:Total 2.4 MB/s | 954 kB 00:00 2023-01-24T19:14:37.694 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:37.714 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:37.714 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:37.810 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:37.813 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:37.899 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:14:37.998 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:38.293 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-24T19:14:38.381 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:14:38.382 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:38.382 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:38.382 INFO:teuthology.orchestra.run.smithi187.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:38.382 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:38.382 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:38.515 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:14:38.570 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install librados-devel 2023-01-24T19:14:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T19:14:38.929 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T19:14:39.313 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:26 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:39.403 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-24T19:14:39.404 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:39.404 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:39.404 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-24T19:14:39.404 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:39.405 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:39.655 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install ceph-volume 2023-01-24T19:14:40.404 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:27 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:40.454 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:40.454 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:40.455 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repo Size 2023-01-24T19:14:40.455 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:40.455 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:14:40.455 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T19:14:40.455 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:40.455 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:14:40.456 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:40.456 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:14:40.456 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:40.457 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 130 k 2023-01-24T19:14:40.457 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 464 k 2023-01-24T19:14:40.457 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:14:40.757 INFO:teuthology.orchestra.run.smithi187.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 431 kB/s | 130 kB 00:00 2023-01-24T19:14:40.758 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:40.758 INFO:teuthology.orchestra.run.smithi187.stdout:Total 429 kB/s | 130 kB 00:00 2023-01-24T19:14:40.759 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:14:40.763 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:14:40.764 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:14:40.816 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:14:40.819 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:14:40.984 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:41.172 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:41.480 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:41.557 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:41.558 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:41.558 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repository Size 2023-01-24T19:14:41.558 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:41.559 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T19:14:41.559 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:41.559 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:41.559 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:41.559 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:14:41.560 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:41.560 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 263 k 2023-01-24T19:14:41.560 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 1.3 M 2023-01-24T19:14:41.560 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:41.960 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:41.961 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:41.961 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:41.961 INFO:teuthology.orchestra.run.smithi187.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:14:41.961 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:41.961 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:42.048 INFO:teuthology.orchestra.run.smithi161.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 536 kB/s | 263 kB 00:00 2023-01-24T19:14:42.049 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:42.050 INFO:teuthology.orchestra.run.smithi161.stdout:Total 535 kB/s | 263 kB 00:00 2023-01-24T19:14:42.050 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:42.060 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:42.060 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:42.165 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:42.169 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:42.355 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:42.608 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:14:43.073 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:14:43.552 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T19:14:43.553 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:43.553 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:43.553 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T19:14:43.553 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:43.553 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:43.725 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install libcephfs2 2023-01-24T19:14:43.780 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install librados-devel 2023-01-24T19:14:44.465 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:31 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:44.516 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:31 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:45.506 INFO:teuthology.orchestra.run.smithi187.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:45.605 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:45.606 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:14:45.606 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:45.672 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:45.672 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:45.672 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2023-01-24T19:14:45.673 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:45.673 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:45.673 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T19:14:45.673 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:45.673 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:45.674 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:45.674 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:14:45.674 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:45.675 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 130 k 2023-01-24T19:14:45.675 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 464 k 2023-01-24T19:14:45.675 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:45.701 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install libcephfs-devel 2023-01-24T19:14:45.966 INFO:teuthology.orchestra.run.smithi161.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 444 kB/s | 130 kB 00:00 2023-01-24T19:14:45.967 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:45.967 INFO:teuthology.orchestra.run.smithi161.stdout:Total 442 kB/s | 130 kB 00:00 2023-01-24T19:14:45.967 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:45.972 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:45.972 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:46.024 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:46.028 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:46.152 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:46.396 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:46.423 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:33 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:46.696 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:47.193 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:47.194 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:14:47.194 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:47.194 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:47.431 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install libcephfs2 2023-01-24T19:14:47.573 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:47.574 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repo Size 2023-01-24T19:14:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:47.574 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:14:47.575 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T19:14:47.575 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:47.575 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:14:47.575 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:14:47.575 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:14:47.576 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:47.576 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 25 k 2023-01-24T19:14:47.576 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 109 k 2023-01-24T19:14:47.576 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:14:47.731 INFO:teuthology.orchestra.run.smithi187.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 158 kB/s | 25 kB 00:00 2023-01-24T19:14:47.732 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:47.732 INFO:teuthology.orchestra.run.smithi187.stdout:Total 157 kB/s | 25 kB 00:00 2023-01-24T19:14:47.733 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:14:47.736 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:14:47.736 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:14:47.737 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:14:47.741 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:14:47.796 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:14:48.005 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:48.165 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:35 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:48.270 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:48.722 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:48.722 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:48.722 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:14:48.723 INFO:teuthology.orchestra.run.smithi187.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:14:48.723 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:14:48.723 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:48.832 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install librados2 2023-01-24T19:14:49.207 INFO:teuthology.orchestra.run.smithi161.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:49.307 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:49.308 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:14:49.308 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:49.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install libcephfs-devel 2023-01-24T19:14:49.561 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:36 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:50.146 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:37 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:50.610 INFO:teuthology.orchestra.run.smithi187.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:50.710 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:50.711 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:14:50.712 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:50.801 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install librbd1 2023-01-24T19:14:51.279 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:51.280 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:51.280 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2023-01-24T19:14:51.280 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:51.280 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:14:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T19:14:51.281 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:51.281 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:14:51.281 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:14:51.281 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:14:51.282 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:51.282 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 25 k 2023-01-24T19:14:51.282 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 109 k 2023-01-24T19:14:51.282 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:14:51.456 INFO:teuthology.orchestra.run.smithi161.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 142 kB/s | 25 kB 00:00 2023-01-24T19:14:51.457 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:14:51.457 INFO:teuthology.orchestra.run.smithi161.stdout:Total 141 kB/s | 25 kB 00:00 2023-01-24T19:14:51.457 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:14:51.460 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:14:51.460 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:14:51.462 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:14:51.465 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:14:51.517 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:14:51.532 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:38 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:51.745 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:52.025 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:14:52.503 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:52.504 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:14:52.504 INFO:teuthology.orchestra.run.smithi161.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:14:52.504 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:14:52.504 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:52.579 INFO:teuthology.orchestra.run.smithi187.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:52.625 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install librados2 2023-01-24T19:14:52.681 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:52.682 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:14:52.683 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:52.774 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install python3-rados 2023-01-24T19:14:53.349 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:40 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:53.493 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:40 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:54.396 INFO:teuthology.orchestra.run.smithi161.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:54.495 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:54.496 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:14:54.496 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:54.535 INFO:teuthology.orchestra.run.smithi187.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:54.607 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install librbd1 2023-01-24T19:14:54.635 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:54.637 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:14:54.637 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:54.738 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install python3-rgw 2023-01-24T19:14:55.342 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:42 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:55.471 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:42 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:56.383 INFO:teuthology.orchestra.run.smithi161.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:56.484 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:56.485 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:14:56.485 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:56.519 INFO:teuthology.orchestra.run.smithi187.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:56.587 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-rados 2023-01-24T19:14:56.620 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:56.621 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:14:56.621 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:56.721 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install python3-cephfs 2023-01-24T19:14:57.309 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:44 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:57.443 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:44 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:58.372 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:58.473 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:14:58.475 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:14:58.475 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:14:58.488 INFO:teuthology.orchestra.run.smithi187.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:14:58.575 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-rgw 2023-01-24T19:14:58.588 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:14:58.589 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:14:58.590 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:14:58.686 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install python3-rbd 2023-01-24T19:14:59.301 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:46 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:14:59.414 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:46 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:00.343 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:15:00.443 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:15:00.444 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:15:00.444 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:15:00.471 INFO:teuthology.orchestra.run.smithi187.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:15:00.568 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-cephfs 2023-01-24T19:15:00.573 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:15:00.574 INFO:teuthology.orchestra.run.smithi187.stdout:Nothing to do. 2023-01-24T19:15:00.575 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:15:00.781 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install rbd-fuse 2023-01-24T19:15:01.288 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:48 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:01.504 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:48 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:02.330 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:15:02.430 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:15:02.431 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:15:02.431 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:15:02.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install python3-rbd 2023-01-24T19:15:02.632 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:15:02.633 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:02.633 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2023-01-24T19:15:02.633 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:02.634 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:15:02.634 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T19:15:02.634 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:02.634 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:15:02.634 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:02.635 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:15:02.635 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:02.635 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 90 k 2023-01-24T19:15:02.635 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 264 k 2023-01-24T19:15:02.635 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:15:02.822 INFO:teuthology.orchestra.run.smithi187.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 481 kB/s | 90 kB 00:00 2023-01-24T19:15:02.822 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:15:02.823 INFO:teuthology.orchestra.run.smithi187.stdout:Total 478 kB/s | 90 kB 00:00 2023-01-24T19:15:02.823 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:15:02.832 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:15:02.832 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:15:02.884 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:15:02.887 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:15:02.999 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:15:03.235 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:03.263 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:50 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:03.518 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:04.024 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:04.025 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:04.025 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:15:04.025 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:15:04.025 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:04.025 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:15:04.230 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install rbd-mirror 2023-01-24T19:15:04.294 INFO:teuthology.orchestra.run.smithi161.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T19:15:04.393 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:15:04.394 INFO:teuthology.orchestra.run.smithi161.stdout:Nothing to do. 2023-01-24T19:15:04.395 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:15:04.498 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install rbd-fuse 2023-01-24T19:15:04.961 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:51 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:05.232 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:52 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:06.092 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:15:06.093 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:06.093 INFO:teuthology.orchestra.run.smithi187.stdout: Package Arch Version Repo Size 2023-01-24T19:15:06.093 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:06.093 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:15:06.094 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:15:06.094 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:06.094 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:15:06.094 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:06.094 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:15:06.094 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:06.095 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 3.5 M 2023-01-24T19:15:06.095 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 16 M 2023-01-24T19:15:06.095 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:15:06.374 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:15:06.375 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:06.375 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2023-01-24T19:15:06.376 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:06.376 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:15:06.376 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T19:15:06.376 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:06.376 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:15:06.377 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:06.377 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:15:06.377 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:06.377 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 90 k 2023-01-24T19:15:06.377 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 264 k 2023-01-24T19:15:06.378 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:15:06.586 INFO:teuthology.orchestra.run.smithi161.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 430 kB/s | 90 kB 00:00 2023-01-24T19:15:06.587 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:15:06.587 INFO:teuthology.orchestra.run.smithi161.stdout:Total 427 kB/s | 90 kB 00:00 2023-01-24T19:15:06.587 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:15:06.597 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:15:06.597 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:15:06.648 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:15:06.652 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:15:06.769 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:15:06.900 INFO:teuthology.orchestra.run.smithi187.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 4.3 MB/s | 3.5 MB 00:00 2023-01-24T19:15:06.901 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:15:06.901 INFO:teuthology.orchestra.run.smithi187.stdout:Total 4.3 MB/s | 3.5 MB 00:00 2023-01-24T19:15:06.902 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:15:06.912 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:15:06.913 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:15:06.968 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:06.975 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:15:06.979 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:15:07.271 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:07.422 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:15:07.637 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:07.798 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:07.799 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:07.799 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:15:07.799 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:15:07.799 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:07.799 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:15:07.999 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install rbd-mirror 2023-01-24T19:15:08.090 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:08.564 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:08.565 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:08.565 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:15:08.565 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:15:08.565 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:08.565 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:15:08.729 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:02:55 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:08.761 DEBUG:teuthology.orchestra.run.smithi187:> sudo yum -y install rbd-nbd 2023-01-24T19:15:09.501 INFO:teuthology.orchestra.run.smithi187.stdout:Last metadata expiration check: 0:02:56 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:09.878 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:15:09.879 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:09.879 INFO:teuthology.orchestra.run.smithi161.stdout: Package Arch Version Repo Size 2023-01-24T19:15:09.879 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:09.880 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:15:09.880 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T19:15:09.880 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:09.880 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:15:09.880 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:09.881 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:15:09.881 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:09.881 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 3.5 M 2023-01-24T19:15:09.881 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 16 M 2023-01-24T19:15:09.881 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:15:10.573 INFO:teuthology.orchestra.run.smithi161.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 5.0 MB/s | 3.5 MB 00:00 2023-01-24T19:15:10.573 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:15:10.574 INFO:teuthology.orchestra.run.smithi161.stdout:Total 5.0 MB/s | 3.5 MB 00:00 2023-01-24T19:15:10.574 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:15:10.585 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:15:10.585 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:15:10.647 INFO:teuthology.orchestra.run.smithi187.stdout:Dependencies resolved. 2023-01-24T19:15:10.648 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:15:10.649 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:10.649 INFO:teuthology.orchestra.run.smithi187.stdout: Package Architecture Version Repository Size 2023-01-24T19:15:10.649 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:10.649 INFO:teuthology.orchestra.run.smithi187.stdout:Installing: 2023-01-24T19:15:10.649 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T19:15:10.650 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:10.650 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction Summary 2023-01-24T19:15:10.650 INFO:teuthology.orchestra.run.smithi187.stdout:================================================================================ 2023-01-24T19:15:10.650 INFO:teuthology.orchestra.run.smithi187.stdout:Install 1 Package 2023-01-24T19:15:10.650 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:10.651 INFO:teuthology.orchestra.run.smithi187.stdout:Total download size: 172 k 2023-01-24T19:15:10.651 INFO:teuthology.orchestra.run.smithi187.stdout:Installed size: 548 k 2023-01-24T19:15:10.651 INFO:teuthology.orchestra.run.smithi187.stdout:Downloading Packages: 2023-01-24T19:15:10.651 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:15:10.937 INFO:teuthology.orchestra.run.smithi187.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 598 kB/s | 172 kB 00:00 2023-01-24T19:15:10.938 INFO:teuthology.orchestra.run.smithi187.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:15:10.938 INFO:teuthology.orchestra.run.smithi187.stdout:Total 595 kB/s | 172 kB 00:00 2023-01-24T19:15:10.939 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction check 2023-01-24T19:15:10.948 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction check succeeded. 2023-01-24T19:15:10.949 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction test 2023-01-24T19:15:10.999 INFO:teuthology.orchestra.run.smithi187.stdout:Transaction test succeeded. 2023-01-24T19:15:11.003 INFO:teuthology.orchestra.run.smithi187.stdout:Running transaction 2023-01-24T19:15:11.094 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:15:11.122 INFO:teuthology.orchestra.run.smithi187.stdout: Preparing : 1/1 2023-01-24T19:15:11.304 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:11.317 INFO:teuthology.orchestra.run.smithi187.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:11.621 INFO:teuthology.orchestra.run.smithi187.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:11.751 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:12.229 INFO:teuthology.orchestra.run.smithi187.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:12.230 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:12.230 INFO:teuthology.orchestra.run.smithi187.stdout:Installed: 2023-01-24T19:15:12.230 INFO:teuthology.orchestra.run.smithi187.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:15:12.230 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:12.230 INFO:teuthology.orchestra.run.smithi187.stdout:Complete! 2023-01-24T19:15:12.316 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:12.316 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:12.317 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:15:12.317 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:15:12.317 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:12.317 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:15:12.430 DEBUG:teuthology.parallel:result is None 2023-01-24T19:15:12.568 DEBUG:teuthology.orchestra.run.smithi161:> sudo yum -y install rbd-nbd 2023-01-24T19:15:13.319 INFO:teuthology.orchestra.run.smithi161.stdout:Last metadata expiration check: 0:03:00 ago on Tue 24 Jan 2023 07:12:13 PM UTC. 2023-01-24T19:15:14.449 INFO:teuthology.orchestra.run.smithi161.stdout:Dependencies resolved. 2023-01-24T19:15:14.450 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:14.451 INFO:teuthology.orchestra.run.smithi161.stdout: Package Architecture Version Repository Size 2023-01-24T19:15:14.451 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:14.451 INFO:teuthology.orchestra.run.smithi161.stdout:Installing: 2023-01-24T19:15:14.451 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T19:15:14.451 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:14.452 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction Summary 2023-01-24T19:15:14.452 INFO:teuthology.orchestra.run.smithi161.stdout:================================================================================ 2023-01-24T19:15:14.452 INFO:teuthology.orchestra.run.smithi161.stdout:Install 1 Package 2023-01-24T19:15:14.452 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:14.452 INFO:teuthology.orchestra.run.smithi161.stdout:Total download size: 172 k 2023-01-24T19:15:14.453 INFO:teuthology.orchestra.run.smithi161.stdout:Installed size: 548 k 2023-01-24T19:15:14.453 INFO:teuthology.orchestra.run.smithi161.stdout:Downloading Packages: 2023-01-24T19:15:14.660 INFO:teuthology.orchestra.run.smithi161.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 828 kB/s | 172 kB 00:00 2023-01-24T19:15:14.661 INFO:teuthology.orchestra.run.smithi161.stdout:-------------------------------------------------------------------------------- 2023-01-24T19:15:14.661 INFO:teuthology.orchestra.run.smithi161.stdout:Total 822 kB/s | 172 kB 00:00 2023-01-24T19:15:14.661 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction check 2023-01-24T19:15:14.671 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction check succeeded. 2023-01-24T19:15:14.671 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction test 2023-01-24T19:15:14.723 INFO:teuthology.orchestra.run.smithi161.stdout:Transaction test succeeded. 2023-01-24T19:15:14.726 INFO:teuthology.orchestra.run.smithi161.stdout:Running transaction 2023-01-24T19:15:14.843 INFO:teuthology.orchestra.run.smithi161.stdout: Preparing : 1/1 2023-01-24T19:15:15.050 INFO:teuthology.orchestra.run.smithi161.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:15.343 INFO:teuthology.orchestra.run.smithi161.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:15.836 INFO:teuthology.orchestra.run.smithi161.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T19:15:15.836 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:15.836 INFO:teuthology.orchestra.run.smithi161.stdout:Installed: 2023-01-24T19:15:15.837 INFO:teuthology.orchestra.run.smithi161.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T19:15:15.837 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:15.837 INFO:teuthology.orchestra.run.smithi161.stdout:Complete! 2023-01-24T19:15:16.081 DEBUG:teuthology.parallel:result is None 2023-01-24T19:15:16.081 DEBUG:teuthology.orchestra.run.smithi161:> sudo lsb_release -is 2023-01-24T19:15:16.118 INFO:teuthology.orchestra.run.smithi161.stdout:CentOSStream 2023-01-24T19:15:16.120 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T19:15:16.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:15:16.269 DEBUG:teuthology.orchestra.run.smithi161:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T19:15:16.292 INFO:teuthology.orchestra.run.smithi161.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T19:15:16.293 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T19:15:16.293 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T19:15:16.293 DEBUG:teuthology.orchestra.run.smithi187:> sudo lsb_release -is 2023-01-24T19:15:16.336 INFO:teuthology.orchestra.run.smithi187.stdout:CentOSStream 2023-01-24T19:15:16.337 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T19:15:16.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T19:15:16.487 DEBUG:teuthology.orchestra.run.smithi187:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T19:15:16.510 INFO:teuthology.orchestra.run.smithi187.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T19:15:16.510 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T19:15:16.511 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T19:15:16.511 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T19:15:16.512 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:16.512 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T19:15:16.541 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:16.541 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T19:15:16.578 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T19:15:16.579 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:16.579 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T19:15:16.609 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T19:15:16.675 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:16.676 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T19:15:16.704 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T19:15:16.771 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T19:15:16.771 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:16.772 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T19:15:16.799 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T19:15:16.867 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:16.868 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T19:15:16.895 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T19:15:16.963 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T19:15:17.025 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T19:15:17.026 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2023-01-24T19:15:17.028 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 777 /var/log/ceph 2023-01-24T19:15:17.055 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T19:15:17.056 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T19:15:17.096 DEBUG:teuthology.orchestra.run.smithi187:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T19:15:17.123 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T19:15:17.123 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T19:15:17.163 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T19:15:17.222 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T19:15:17.222 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-24T19:15:17.222 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135639', 'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'description': 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/cache-snaps}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7135639', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'targets': {'smithi161.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCy1aqx1Mh+njlc7fQQEYxU88GQgNFDqWaTrIUB26LOyZlUOJXMTvLATxVditSbnexN+kyO3vZtRr2ZWK6GgOJhlgfUgKhDwRTAuUH8Wlehgs514F8n07nx5mrJKOj4rjypXiJjKV3FF1T8vh8yo2cpre2+pZXQIEDCdosyfcpfXyaEtPud1WD10u4xUAgY3SuC+66Tu/74e1O4waHyrA38O8Z151/EhD61J+frT9b2WWrI3wro0le4p6p/6PZ4z45fOaJRaNJ2pOQE+Yy+pwvrYQjQ7Y8Jy8CJYFpz9wAI5YypUquWvvv+xhYSIfOiX3gilcxqgqh/PMqCtcVcxgdKTaFp+6gKLiXtrlygXZEAFh8hUdIBGS0574MNrobpWG7Wl/lTLTaGpcz8QDz5h/mNdBJTrzaZenJWPA47bfdve6vo39C6zgjOKXyfMbv2xHmpHE4bSjY54SDwIr6l3aDt0imkT22hF+UMAIEFR1iZ7h0bAIoA3kCxMSNe9UbLEZE=', 'smithi187.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCbyX5Dx+eTTEL2QiGonHxt7lJUb0XN8i05S7zE3vxj0g1y+5Nr9ZP5VszpOBiDUeyBqeSINlkqtgOt7fr8BQeu9KQDDeKuOFvjci/NffpCtFgFFny8+Yb8ymxn4Iug63mYao/2z2viBivF9zNKAYfFBC7vLTpZ5QcY3R28v5ivLNaRsseziSvDtSzOZcm2BmEYcZJLuz9vvxo6Hbkh19WgSAaE6pLLJsMfNRJofuASFJ6urMOy0vnIed8RWCUSwSuVswZnvhhAKgBPLxwdOrlJ8Xodup7HvYXsteFyKBqHW0avZYGgx6GufC4IWE/j66FYAdXD97rKOEDtvMxUYiTe4ARyE0c73NO2/lq85jSxaTqeZym13Vks7OCdS1eAZNiSPfRKJc71+JTUmbGCEWBlKAxu5lQede6M+JfiN/t6rbhvthGSl/OH3MYsL5e1V7e5B4osxrBzThWPT4OXfQIVW4Ee47i/fRGSmR9M09yOm76MxJ1dItfdCHhl7Stv9/U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'global': {'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(REQUEST_STUCK\\)', 'failed to encode map', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'must scrub before tier agent can activate'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'cluster': 'ceph'}}, {'exec': {'mon.a': ['while ! ceph balancer status ; do sleep 1 ; done', 'ceph balancer mode crush-compat', 'ceph balancer on']}}, {'exec': {'client.0': ['sudo ceph osd pool create base 4', 'sudo ceph osd pool application enable base rados', 'sudo ceph osd pool create cache 4', 'sudo ceph osd tier add base cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay base cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 3600', 'sudo ceph osd pool set cache target_max_objects 250', 'sudo ceph osd pool set cache min_read_recency_for_promote 2']}}, {'rados': {'clients': ['client.0'], 'objects': 500, 'op_weights': {'cache_evict': 50, 'cache_flush': 50, 'cache_try_flush': 50, 'copy_from': 50, 'delete': 50, 'read': 100, 'rollback': 50, 'snap_create': 50, 'snap_remove': 50, 'write': 100}, 'ops': 4000, 'pools': ['base']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_16:05:13', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T19:15:17.223 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T19:15:17.231 DEBUG:teuthology.orchestra.run.smithi187:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T19:15:17.249 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T19:15:17.291 DEBUG:teuthology.orchestra.run.smithi187:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T19:15:17.318 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:17.319 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T19:15:17.374 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T19:15:17.375 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2023-01-24T19:15:17.433 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T19:15:17.433 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:17.434 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 371008 Links: 1 2023-01-24T19:15:17.434 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:17.434 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:17.434 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-01-24 19:15:11.485154161 +0000 2023-01-24T19:15:17.434 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-01-24 19:11:14.549999784 +0000 2023-01-24T19:15:17.435 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-01-24 19:11:14.549999784 +0000 2023-01-24T19:15:17.435 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-01-24T19:15:17.435 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T19:15:17.500 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-01-24T19:15:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-01-24T19:15:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000230345 s, 2.2 MB/s 2023-01-24T19:15:17.502 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T19:15:17.562 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2023-01-24T19:15:17.620 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T19:15:17.620 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:17.620 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 369408 Links: 1 2023-01-24T19:15:17.621 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:17.621 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:17.621 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-01-24 19:15:11.485154161 +0000 2023-01-24T19:15:17.621 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-01-24 19:11:15.062989332 +0000 2023-01-24T19:15:17.621 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-01-24 19:11:15.062989332 +0000 2023-01-24T19:15:17.622 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-01-24T19:15:17.622 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T19:15:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-01-24T19:15:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-01-24T19:15:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000219114 s, 2.3 MB/s 2023-01-24T19:15:17.687 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T19:15:17.746 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2023-01-24T19:15:17.803 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T19:15:17.803 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:17.804 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 373190 Links: 1 2023-01-24T19:15:17.804 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:17.804 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:17.804 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-01-24 19:15:11.485154161 +0000 2023-01-24T19:15:17.805 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-01-24 19:11:15.588978616 +0000 2023-01-24T19:15:17.805 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-01-24 19:11:15.588978616 +0000 2023-01-24T19:15:17.805 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-01-24T19:15:17.805 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T19:15:17.869 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-01-24T19:15:17.870 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-01-24T19:15:17.870 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000217589 s, 2.4 MB/s 2023-01-24T19:15:17.871 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T19:15:17.929 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2023-01-24T19:15:17.987 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T19:15:17.988 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:17.988 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 6h/6d Inode: 372405 Links: 1 2023-01-24T19:15:17.988 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:17.988 INFO:teuthology.orchestra.run.smithi161.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:17.988 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-01-24 19:15:11.485154161 +0000 2023-01-24T19:15:17.989 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-01-24 19:11:16.085968490 +0000 2023-01-24T19:15:17.989 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-01-24 19:11:16.085968490 +0000 2023-01-24T19:15:17.989 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-01-24T19:15:17.989 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T19:15:18.053 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-01-24T19:15:18.053 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-01-24T19:15:18.053 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000208104 s, 2.5 MB/s 2023-01-24T19:15:18.054 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T19:15:18.113 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:18.114 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:18.114 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T19:15:18.132 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T19:15:18.133 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_1 2023-01-24T19:15:18.190 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T19:15:18.191 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:18.191 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 375833 Links: 1 2023-01-24T19:15:18.191 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:18.191 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:18.192 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-01-24 19:15:07.822351287 +0000 2023-01-24T19:15:18.192 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-01-24 19:11:20.667204020 +0000 2023-01-24T19:15:18.192 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-01-24 19:11:20.667204020 +0000 2023-01-24T19:15:18.192 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-01-24T19:15:18.193 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T19:15:18.256 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-01-24T19:15:18.257 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-01-24T19:15:18.257 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000213649 s, 2.4 MB/s 2023-01-24T19:15:18.258 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T19:15:18.317 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_2 2023-01-24T19:15:18.375 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T19:15:18.375 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:18.376 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 375881 Links: 1 2023-01-24T19:15:18.376 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:18.376 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:18.376 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-01-24 19:15:07.822351287 +0000 2023-01-24T19:15:18.377 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-01-24 19:11:21.170195817 +0000 2023-01-24T19:15:18.377 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-01-24 19:11:21.170195817 +0000 2023-01-24T19:15:18.377 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-01-24T19:15:18.377 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T19:15:18.441 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-01-24T19:15:18.442 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-01-24T19:15:18.442 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000231569 s, 2.2 MB/s 2023-01-24T19:15:18.443 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T19:15:18.502 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_3 2023-01-24T19:15:18.560 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T19:15:18.561 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:18.561 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 375419 Links: 1 2023-01-24T19:15:18.561 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:18.561 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:18.561 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-01-24 19:15:07.823351271 +0000 2023-01-24T19:15:18.562 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-01-24 19:11:21.687187385 +0000 2023-01-24T19:15:18.562 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-01-24 19:11:21.687187385 +0000 2023-01-24T19:15:18.562 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-01-24T19:15:18.562 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T19:15:18.626 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-01-24T19:15:18.626 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-01-24T19:15:18.627 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000229958 s, 2.2 MB/s 2023-01-24T19:15:18.627 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T19:15:18.686 DEBUG:teuthology.orchestra.run.smithi187:> stat /dev/vg_nvme/lv_4 2023-01-24T19:15:18.744 INFO:teuthology.orchestra.run.smithi187.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T19:15:18.745 INFO:teuthology.orchestra.run.smithi187.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T19:15:18.745 INFO:teuthology.orchestra.run.smithi187.stdout:Device: 6h/6d Inode: 375541 Links: 1 2023-01-24T19:15:18.745 INFO:teuthology.orchestra.run.smithi187.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T19:15:18.745 INFO:teuthology.orchestra.run.smithi187.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T19:15:18.745 INFO:teuthology.orchestra.run.smithi187.stdout:Access: 2023-01-24 19:15:07.823351271 +0000 2023-01-24T19:15:18.746 INFO:teuthology.orchestra.run.smithi187.stdout:Modify: 2023-01-24 19:11:22.201179002 +0000 2023-01-24T19:15:18.746 INFO:teuthology.orchestra.run.smithi187.stdout:Change: 2023-01-24 19:11:22.201179002 +0000 2023-01-24T19:15:18.746 INFO:teuthology.orchestra.run.smithi187.stdout: Birth: - 2023-01-24T19:15:18.746 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T19:15:18.810 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records in 2023-01-24T19:15:18.810 INFO:teuthology.orchestra.run.smithi187.stderr:1+0 records out 2023-01-24T19:15:18.810 INFO:teuthology.orchestra.run.smithi187.stderr:512 bytes copied, 0.000206295 s, 2.5 MB/s 2023-01-24T19:15:18.811 DEBUG:teuthology.orchestra.run.smithi187:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T19:15:18.870 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:18.870 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T19:15:18.871 INFO:tasks.ceph:Generating config... 2023-01-24T19:15:18.872 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-24T19:15:18.872 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-01-24T19:15:18.873 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2023-01-24T19:15:18.873 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-01-24T19:15:18.873 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-01-24T19:15:18.873 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-24T19:15:18.874 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-01-24T19:15:18.874 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-01-24T19:15:18.874 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T19:15:18.874 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T19:15:18.875 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-01-24T19:15:18.875 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-01-24T19:15:18.875 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-01-24T19:15:18.876 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-01-24T19:15:18.876 INFO:tasks.ceph:[osd] osd backoff on peering = True 2023-01-24T19:15:18.876 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-24T19:15:18.876 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-24T19:15:18.877 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-01-24T19:15:18.877 INFO:tasks.ceph:[osd] osd heartbeat use min delay socket = True 2023-01-24T19:15:18.877 INFO:tasks.ceph:[osd] osd inject bad map crc probability = 0.1 2023-01-24T19:15:18.877 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2023-01-24T19:15:18.878 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-24T19:15:18.878 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-01-24T19:15:18.878 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-24T19:15:18.878 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-24T19:15:18.879 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-24T19:15:18.879 INFO:tasks.ceph:[osd] osd_max_pg_log_entries = 3000 2023-01-24T19:15:18.879 INFO:tasks.ceph:[osd] osd_min_pg_log_entries = 3000 2023-01-24T19:15:18.879 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-01-24T19:15:18.880 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-01-24T19:15:18.880 INFO:tasks.ceph:[global] ms bind msgr1 = False 2023-01-24T19:15:18.880 INFO:tasks.ceph:[global] ms bind msgr2 = True 2023-01-24T19:15:18.880 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-01-24T19:15:18.881 INFO:tasks.ceph:[global] ms type = async 2023-01-24T19:15:18.881 INFO:tasks.ceph:[global] osd_pool_default_min_size = 2 2023-01-24T19:15:18.881 INFO:tasks.ceph:[global] osd_pool_default_size = 2 2023-01-24T19:15:18.882 INFO:tasks.ceph:[global] osd_recovery_max_active = 10 2023-01-24T19:15:18.882 INFO:tasks.ceph:[global] osd_recovery_max_single_start = 10 2023-01-24T19:15:18.882 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T19:15:18.882 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T19:15:18.883 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T19:15:18.883 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T19:15:18.883 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T19:15:18.883 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-24T19:15:18.884 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T19:15:18.884 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T19:15:18.936 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T19:15:18.938 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T19:15:18.982 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T19:15:19.049 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.161'), ('mon.c', '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]'), ('mon.b', '172.21.15.187')] 2023-01-24T19:15:19.050 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790],172.21.15.187', 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async', 'osd_pool_default_min_size': 2, 'osd_pool_default_size': 2, 'osd_recovery_max_active': 10, 'osd_recovery_max_single_start': 10}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd backoff on peering': True, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd heartbeat use min delay socket': True, 'osd inject bad map crc probability': 0.1, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True, 'osd_max_pg_log_entries': 3000, 'osd_min_pg_log_entries': 3000}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2023-01-24T19:15:19.051 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:19.051 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T19:15:19.107 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.161 --addv c '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]' --add b 172.21.15.187 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:19.193 INFO:teuthology.orchestra.run.smithi161.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:19.193 INFO:teuthology.orchestra.run.smithi161.stdout:monmaptool: generated fsid 2fd00fc0-0054-482e-9800-3bb42aed5401 2023-01-24T19:15:19.194 INFO:teuthology.orchestra.run.smithi161.stdout:setting min_mon_release = octopus 2023-01-24T19:15:19.194 INFO:teuthology.orchestra.run.smithi161.stdout:epoch 0 2023-01-24T19:15:19.194 INFO:teuthology.orchestra.run.smithi161.stdout:fsid 2fd00fc0-0054-482e-9800-3bb42aed5401 2023-01-24T19:15:19.194 INFO:teuthology.orchestra.run.smithi161.stdout:last_changed 2023-01-24T19:15:19.193169+0000 2023-01-24T19:15:19.194 INFO:teuthology.orchestra.run.smithi161.stdout:created 2023-01-24T19:15:19.193169+0000 2023-01-24T19:15:19.195 INFO:teuthology.orchestra.run.smithi161.stdout:min_mon_release 15 (octopus) 2023-01-24T19:15:19.195 INFO:teuthology.orchestra.run.smithi161.stdout:election_strategy: 1 2023-01-24T19:15:19.195 INFO:teuthology.orchestra.run.smithi161.stdout:0: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.a 2023-01-24T19:15:19.196 INFO:teuthology.orchestra.run.smithi161.stdout:1: [v2:172.21.15.187:3300/0,v1:172.21.15.187:6789/0] mon.b 2023-01-24T19:15:19.196 INFO:teuthology.orchestra.run.smithi161.stdout:2: [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] mon.c 2023-01-24T19:15:19.196 INFO:teuthology.orchestra.run.smithi161.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-01-24T19:15:19.197 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T19:15:19.215 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2fd00fc0-0054-482e-9800-3bb42aed5401... 2023-01-24T19:15:19.217 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T19:15:19.258 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T19:15:19.301 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2023-01-24T19:15:19.301 INFO:teuthology.orchestra.run.smithi161.stdout: chdir = "" 2023-01-24T19:15:19.301 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T19:15:19.302 INFO:teuthology.orchestra.run.smithi161.stdout: auth supported = cephx 2023-01-24T19:15:19.302 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.302 INFO:teuthology.orchestra.run.smithi161.stdout: filestore xattr use omap = true 2023-01-24T19:15:19.302 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.302 INFO:teuthology.orchestra.run.smithi161.stdout: mon clock drift allowed = 1.000 2023-01-24T19:15:19.303 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.303 INFO:teuthology.orchestra.run.smithi161.stdout: osd crush chooseleaf type = 0 2023-01-24T19:15:19.303 INFO:teuthology.orchestra.run.smithi161.stdout: auth debug = true 2023-01-24T19:15:19.303 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.304 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on old message = true 2023-01-24T19:15:19.304 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on bug = true 2023-01-24T19:15:19.304 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.304 INFO:teuthology.orchestra.run.smithi161.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T19:15:19.304 INFO:teuthology.orchestra.run.smithi161.stdout: mon pg warn max object skew = 0 2023-01-24T19:15:19.305 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.305 INFO:teuthology.orchestra.run.smithi161.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T19:15:19.305 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T19:15:19.305 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.305 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default size = 2 2023-01-24T19:15:19.305 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.306 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow primary affinity = true 2023-01-24T19:15:19.306 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow pg remap = true 2023-01-24T19:15:19.306 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on legacy crush tunables = false 2023-01-24T19:15:19.306 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on crush straw calc version zero = false 2023-01-24T19:15:19.306 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on no sortbitwise = false 2023-01-24T19:15:19.307 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on osd down out interval zero = false 2023-01-24T19:15:19.307 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on too few osds = false 2023-01-24T19:15:19.307 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T19:15:19.307 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T19:15:19.307 INFO:teuthology.orchestra.run.smithi161.stdout: mon_allow_pool_size_one = true 2023-01-24T19:15:19.308 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.308 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T19:15:19.308 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.308 INFO:teuthology.orchestra.run.smithi161.stdout: osd default data pool replay window = 5 2023-01-24T19:15:19.308 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.309 INFO:teuthology.orchestra.run.smithi161.stdout: mon allow pool delete = true 2023-01-24T19:15:19.309 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.309 INFO:teuthology.orchestra.run.smithi161.stdout: mon cluster log file level = debug 2023-01-24T19:15:19.309 INFO:teuthology.orchestra.run.smithi161.stdout: debug asserts on shutdown = true 2023-01-24T19:15:19.309 INFO:teuthology.orchestra.run.smithi161.stdout: mon health detail to clog = false 2023-01-24T19:15:19.310 INFO:teuthology.orchestra.run.smithi161.stdout: mon host = "172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790],172.21.15.187" 2023-01-24T19:15:19.310 INFO:teuthology.orchestra.run.smithi161.stdout: mon client directed command retry = 5 2023-01-24T19:15:19.310 INFO:teuthology.orchestra.run.smithi161.stdout: mon election default strategy = 1 2023-01-24T19:15:19.310 INFO:teuthology.orchestra.run.smithi161.stdout: ms bind msgr1 = False 2023-01-24T19:15:19.310 INFO:teuthology.orchestra.run.smithi161.stdout: ms bind msgr2 = True 2023-01-24T19:15:19.311 INFO:teuthology.orchestra.run.smithi161.stdout: ms inject socket failures = 5000 2023-01-24T19:15:19.311 INFO:teuthology.orchestra.run.smithi161.stdout: ms type = async 2023-01-24T19:15:19.311 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_min_size = 2 2023-01-24T19:15:19.311 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_size = 2 2023-01-24T19:15:19.311 INFO:teuthology.orchestra.run.smithi161.stdout: osd_recovery_max_active = 10 2023-01-24T19:15:19.311 INFO:teuthology.orchestra.run.smithi161.stdout: osd_recovery_max_single_start = 10 2023-01-24T19:15:19.312 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = 2fd00fc0-0054-482e-9800-3bb42aed5401 2023-01-24T19:15:19.312 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.312 INFO:teuthology.orchestra.run.smithi161.stdout:[osd] 2023-01-24T19:15:19.312 INFO:teuthology.orchestra.run.smithi161.stdout: osd journal size = 100 2023-01-24T19:15:19.312 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.313 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub load threshold = 5.0 2023-01-24T19:15:19.313 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub max interval = 600 2023-01-24T19:15:19.313 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.313 INFO:teuthology.orchestra.run.smithi161.stdout: osd recover clone overlap = true 2023-01-24T19:15:19.313 INFO:teuthology.orchestra.run.smithi161.stdout: osd recovery max chunk = 1048576 2023-01-24T19:15:19.314 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.314 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug shutdown = true 2023-01-24T19:15:19.314 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug op order = true 2023-01-24T19:15:19.314 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify stray on activate = true 2023-01-24T19:15:19.314 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.315 INFO:teuthology.orchestra.run.smithi161.stdout: osd open classes on start = true 2023-01-24T19:15:19.315 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug pg log writeout = true 2023-01-24T19:15:19.315 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.315 INFO:teuthology.orchestra.run.smithi161.stdout: osd deep scrub update digest min age = 30 2023-01-24T19:15:19.315 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.315 INFO:teuthology.orchestra.run.smithi161.stdout: osd map max advance = 10 2023-01-24T19:15:19.316 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.316 INFO:teuthology.orchestra.run.smithi161.stdout: journal zero on create = true 2023-01-24T19:15:19.316 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.316 INFO:teuthology.orchestra.run.smithi161.stdout: filestore ondisk finisher threads = 3 2023-01-24T19:15:19.316 INFO:teuthology.orchestra.run.smithi161.stdout: filestore apply finisher threads = 3 2023-01-24T19:15:19.317 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.317 INFO:teuthology.orchestra.run.smithi161.stdout: bdev debug aio = true 2023-01-24T19:15:19.317 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug misdirected ops = true 2023-01-24T19:15:19.317 INFO:teuthology.orchestra.run.smithi161.stdout: debug monc = 20 2023-01-24T19:15:19.317 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore block size = 96636764160 2023-01-24T19:15:19.318 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore compression algorithm = zstd 2023-01-24T19:15:19.318 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore compression mode = aggressive 2023-01-24T19:15:19.318 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore fsck on mount = True 2023-01-24T19:15:19.318 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore zero block detection = True 2023-01-24T19:15:19.318 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluefs = 20 2023-01-24T19:15:19.319 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluestore = 20 2023-01-24T19:15:19.319 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-01-24T19:15:19.319 INFO:teuthology.orchestra.run.smithi161.stdout: debug osd = 20 2023-01-24T19:15:19.319 INFO:teuthology.orchestra.run.smithi161.stdout: debug rocksdb = 10 2023-01-24T19:15:19.319 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T19:15:19.319 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.9 2023-01-24T19:15:19.320 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.8 2023-01-24T19:15:19.320 INFO:teuthology.orchestra.run.smithi161.stdout: osd backoff on peering = True 2023-01-24T19:15:19.320 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify cached snaps = True 2023-01-24T19:15:19.320 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify missing on start = True 2023-01-24T19:15:19.320 INFO:teuthology.orchestra.run.smithi161.stdout: osd failsafe full ratio = 0.95 2023-01-24T19:15:19.321 INFO:teuthology.orchestra.run.smithi161.stdout: osd heartbeat use min delay socket = True 2023-01-24T19:15:19.321 INFO:teuthology.orchestra.run.smithi161.stdout: osd inject bad map crc probability = 0.1 2023-01-24T19:15:19.321 INFO:teuthology.orchestra.run.smithi161.stdout: osd max markdown count = 1000 2023-01-24T19:15:19.321 INFO:teuthology.orchestra.run.smithi161.stdout: osd mclock override recovery settings = True 2023-01-24T19:15:19.321 INFO:teuthology.orchestra.run.smithi161.stdout: osd objectstore = bluestore 2023-01-24T19:15:19.322 INFO:teuthology.orchestra.run.smithi161.stdout: osd op queue = debug_random 2023-01-24T19:15:19.322 INFO:teuthology.orchestra.run.smithi161.stdout: osd op queue cut off = debug_random 2023-01-24T19:15:19.322 INFO:teuthology.orchestra.run.smithi161.stdout: osd shutdown pgref assert = True 2023-01-24T19:15:19.322 INFO:teuthology.orchestra.run.smithi161.stdout: osd_max_pg_log_entries = 3000 2023-01-24T19:15:19.322 INFO:teuthology.orchestra.run.smithi161.stdout: osd_min_pg_log_entries = 3000 2023-01-24T19:15:19.322 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.323 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr] 2023-01-24T19:15:19.323 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-01-24T19:15:19.323 INFO:teuthology.orchestra.run.smithi161.stdout: debug mgr = 20 2023-01-24T19:15:19.323 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2023-01-24T19:15:19.323 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2023-01-24T19:15:19.324 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:15:19.324 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:15:19.324 INFO:teuthology.orchestra.run.smithi161.stdout: mgr/telemetry/nag = false 2023-01-24T19:15:19.324 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.324 INFO:teuthology.orchestra.run.smithi161.stdout:[mon] 2023-01-24T19:15:19.325 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-01-24T19:15:19.325 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2023-01-24T19:15:19.325 INFO:teuthology.orchestra.run.smithi161.stdout: debug paxos = 20 2023-01-24T19:15:19.325 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2023-01-24T19:15:19.325 INFO:teuthology.orchestra.run.smithi161.stdout: mon data avail warn = 5 2023-01-24T19:15:19.326 INFO:teuthology.orchestra.run.smithi161.stdout: mon mgr mkfs grace = 240 2023-01-24T19:15:19.326 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:15:19.326 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd reporter subtree level = osd 2023-01-24T19:15:19.326 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd prime pg temp = true 2023-01-24T19:15:19.326 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:15:19.326 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.327 INFO:teuthology.orchestra.run.smithi161.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T19:15:19.327 INFO:teuthology.orchestra.run.smithi161.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T19:15:19.327 INFO:teuthology.orchestra.run.smithi161.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T19:15:19.327 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.327 INFO:teuthology.orchestra.run.smithi161.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T19:15:19.327 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T19:15:19.328 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T19:15:19.328 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.328 INFO:teuthology.orchestra.run.smithi161.stdout: # 1m isn't quite enough 2023-01-24T19:15:19.328 INFO:teuthology.orchestra.run.smithi161.stdout: mon_down_mkfs_grace = 2m 2023-01-24T19:15:19.328 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.329 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_filestore_osds = false 2023-01-24T19:15:19.329 INFO:teuthology.orchestra.run.smithi161.stdout: mon scrub interval = 300 2023-01-24T19:15:19.329 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:15:19.329 INFO:teuthology.orchestra.run.smithi161.stdout:[client] 2023-01-24T19:15:19.329 INFO:teuthology.orchestra.run.smithi161.stdout: rgw cache enabled = true 2023-01-24T19:15:19.330 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable ops log = true 2023-01-24T19:15:19.330 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable usage log = true 2023-01-24T19:15:19.330 INFO:teuthology.orchestra.run.smithi161.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T19:15:19.330 INFO:teuthology.orchestra.run.smithi161.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T19:15:19.330 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.a] 2023-01-24T19:15:19.330 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.c] 2023-01-24T19:15:19.331 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.b] 2023-01-24T19:15:19.332 INFO:teuthology.orchestra.run.smithi187.stdout:[global] 2023-01-24T19:15:19.332 INFO:teuthology.orchestra.run.smithi187.stdout: chdir = "" 2023-01-24T19:15:19.332 INFO:teuthology.orchestra.run.smithi187.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T19:15:19.333 INFO:teuthology.orchestra.run.smithi187.stdout: auth supported = cephx 2023-01-24T19:15:19.333 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.333 INFO:teuthology.orchestra.run.smithi187.stdout: filestore xattr use omap = true 2023-01-24T19:15:19.333 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.333 INFO:teuthology.orchestra.run.smithi187.stdout: mon clock drift allowed = 1.000 2023-01-24T19:15:19.334 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.334 INFO:teuthology.orchestra.run.smithi187.stdout: osd crush chooseleaf type = 0 2023-01-24T19:15:19.334 INFO:teuthology.orchestra.run.smithi187.stdout: auth debug = true 2023-01-24T19:15:19.334 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.334 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on old message = true 2023-01-24T19:15:19.335 INFO:teuthology.orchestra.run.smithi187.stdout: ms die on bug = true 2023-01-24T19:15:19.335 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.335 INFO:teuthology.orchestra.run.smithi187.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T19:15:19.335 INFO:teuthology.orchestra.run.smithi187.stdout: mon pg warn max object skew = 0 2023-01-24T19:15:19.335 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.335 INFO:teuthology.orchestra.run.smithi187.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T19:15:19.336 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T19:15:19.336 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.336 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default size = 2 2023-01-24T19:15:19.336 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow primary affinity = true 2023-01-24T19:15:19.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd allow pg remap = true 2023-01-24T19:15:19.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on legacy crush tunables = false 2023-01-24T19:15:19.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on crush straw calc version zero = false 2023-01-24T19:15:19.337 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on no sortbitwise = false 2023-01-24T19:15:19.338 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on osd down out interval zero = false 2023-01-24T19:15:19.338 INFO:teuthology.orchestra.run.smithi187.stdout: mon warn on too few osds = false 2023-01-24T19:15:19.338 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T19:15:19.338 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T19:15:19.338 INFO:teuthology.orchestra.run.smithi187.stdout: mon_allow_pool_size_one = true 2023-01-24T19:15:19.338 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.339 INFO:teuthology.orchestra.run.smithi187.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T19:15:19.339 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.339 INFO:teuthology.orchestra.run.smithi187.stdout: osd default data pool replay window = 5 2023-01-24T19:15:19.339 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.339 INFO:teuthology.orchestra.run.smithi187.stdout: mon allow pool delete = true 2023-01-24T19:15:19.340 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon cluster log file level = debug 2023-01-24T19:15:19.340 INFO:teuthology.orchestra.run.smithi187.stdout: debug asserts on shutdown = true 2023-01-24T19:15:19.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon health detail to clog = false 2023-01-24T19:15:19.340 INFO:teuthology.orchestra.run.smithi187.stdout: mon host = "172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790],172.21.15.187" 2023-01-24T19:15:19.341 INFO:teuthology.orchestra.run.smithi187.stdout: mon client directed command retry = 5 2023-01-24T19:15:19.341 INFO:teuthology.orchestra.run.smithi187.stdout: mon election default strategy = 1 2023-01-24T19:15:19.341 INFO:teuthology.orchestra.run.smithi187.stdout: ms bind msgr1 = False 2023-01-24T19:15:19.341 INFO:teuthology.orchestra.run.smithi187.stdout: ms bind msgr2 = True 2023-01-24T19:15:19.341 INFO:teuthology.orchestra.run.smithi187.stdout: ms inject socket failures = 5000 2023-01-24T19:15:19.342 INFO:teuthology.orchestra.run.smithi187.stdout: ms type = async 2023-01-24T19:15:19.342 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_min_size = 2 2023-01-24T19:15:19.342 INFO:teuthology.orchestra.run.smithi187.stdout: osd_pool_default_size = 2 2023-01-24T19:15:19.342 INFO:teuthology.orchestra.run.smithi187.stdout: osd_recovery_max_active = 10 2023-01-24T19:15:19.342 INFO:teuthology.orchestra.run.smithi187.stdout: osd_recovery_max_single_start = 10 2023-01-24T19:15:19.342 INFO:teuthology.orchestra.run.smithi187.stdout: fsid = 2fd00fc0-0054-482e-9800-3bb42aed5401 2023-01-24T19:15:19.343 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.343 INFO:teuthology.orchestra.run.smithi187.stdout:[osd] 2023-01-24T19:15:19.343 INFO:teuthology.orchestra.run.smithi187.stdout: osd journal size = 100 2023-01-24T19:15:19.343 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.343 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub load threshold = 5.0 2023-01-24T19:15:19.344 INFO:teuthology.orchestra.run.smithi187.stdout: osd scrub max interval = 600 2023-01-24T19:15:19.344 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.344 INFO:teuthology.orchestra.run.smithi187.stdout: osd recover clone overlap = true 2023-01-24T19:15:19.344 INFO:teuthology.orchestra.run.smithi187.stdout: osd recovery max chunk = 1048576 2023-01-24T19:15:19.344 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.345 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug shutdown = true 2023-01-24T19:15:19.345 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug op order = true 2023-01-24T19:15:19.345 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify stray on activate = true 2023-01-24T19:15:19.345 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.345 INFO:teuthology.orchestra.run.smithi187.stdout: osd open classes on start = true 2023-01-24T19:15:19.346 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug pg log writeout = true 2023-01-24T19:15:19.346 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.346 INFO:teuthology.orchestra.run.smithi187.stdout: osd deep scrub update digest min age = 30 2023-01-24T19:15:19.346 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.346 INFO:teuthology.orchestra.run.smithi187.stdout: osd map max advance = 10 2023-01-24T19:15:19.347 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.347 INFO:teuthology.orchestra.run.smithi187.stdout: journal zero on create = true 2023-01-24T19:15:19.347 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.347 INFO:teuthology.orchestra.run.smithi187.stdout: filestore ondisk finisher threads = 3 2023-01-24T19:15:19.347 INFO:teuthology.orchestra.run.smithi187.stdout: filestore apply finisher threads = 3 2023-01-24T19:15:19.347 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.348 INFO:teuthology.orchestra.run.smithi187.stdout: bdev debug aio = true 2023-01-24T19:15:19.348 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug misdirected ops = true 2023-01-24T19:15:19.348 INFO:teuthology.orchestra.run.smithi187.stdout: debug monc = 20 2023-01-24T19:15:19.348 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore block size = 96636764160 2023-01-24T19:15:19.348 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore compression algorithm = zstd 2023-01-24T19:15:19.349 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore compression mode = aggressive 2023-01-24T19:15:19.349 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore fsck on mount = True 2023-01-24T19:15:19.349 INFO:teuthology.orchestra.run.smithi187.stdout: bluestore zero block detection = True 2023-01-24T19:15:19.349 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluefs = 20 2023-01-24T19:15:19.349 INFO:teuthology.orchestra.run.smithi187.stdout: debug bluestore = 20 2023-01-24T19:15:19.350 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2023-01-24T19:15:19.350 INFO:teuthology.orchestra.run.smithi187.stdout: debug osd = 20 2023-01-24T19:15:19.350 INFO:teuthology.orchestra.run.smithi187.stdout: debug rocksdb = 10 2023-01-24T19:15:19.350 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd backfillfull_ratio = 0.85 2023-01-24T19:15:19.350 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd full ratio = 0.9 2023-01-24T19:15:19.351 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd nearfull ratio = 0.8 2023-01-24T19:15:19.351 INFO:teuthology.orchestra.run.smithi187.stdout: osd backoff on peering = True 2023-01-24T19:15:19.351 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify cached snaps = True 2023-01-24T19:15:19.351 INFO:teuthology.orchestra.run.smithi187.stdout: osd debug verify missing on start = True 2023-01-24T19:15:19.351 INFO:teuthology.orchestra.run.smithi187.stdout: osd failsafe full ratio = 0.95 2023-01-24T19:15:19.351 INFO:teuthology.orchestra.run.smithi187.stdout: osd heartbeat use min delay socket = True 2023-01-24T19:15:19.352 INFO:teuthology.orchestra.run.smithi187.stdout: osd inject bad map crc probability = 0.1 2023-01-24T19:15:19.352 INFO:teuthology.orchestra.run.smithi187.stdout: osd max markdown count = 1000 2023-01-24T19:15:19.352 INFO:teuthology.orchestra.run.smithi187.stdout: osd mclock override recovery settings = True 2023-01-24T19:15:19.352 INFO:teuthology.orchestra.run.smithi187.stdout: osd objectstore = bluestore 2023-01-24T19:15:19.352 INFO:teuthology.orchestra.run.smithi187.stdout: osd op queue = debug_random 2023-01-24T19:15:19.353 INFO:teuthology.orchestra.run.smithi187.stdout: osd op queue cut off = debug_random 2023-01-24T19:15:19.353 INFO:teuthology.orchestra.run.smithi187.stdout: osd shutdown pgref assert = True 2023-01-24T19:15:19.353 INFO:teuthology.orchestra.run.smithi187.stdout: osd_max_pg_log_entries = 3000 2023-01-24T19:15:19.353 INFO:teuthology.orchestra.run.smithi187.stdout: osd_min_pg_log_entries = 3000 2023-01-24T19:15:19.353 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.354 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr] 2023-01-24T19:15:19.354 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2023-01-24T19:15:19.354 INFO:teuthology.orchestra.run.smithi187.stdout: debug mgr = 20 2023-01-24T19:15:19.354 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2023-01-24T19:15:19.354 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2023-01-24T19:15:19.355 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:15:19.355 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:15:19.355 INFO:teuthology.orchestra.run.smithi187.stdout: mgr/telemetry/nag = false 2023-01-24T19:15:19.355 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.355 INFO:teuthology.orchestra.run.smithi187.stdout:[mon] 2023-01-24T19:15:19.356 INFO:teuthology.orchestra.run.smithi187.stdout: debug ms = 1 2023-01-24T19:15:19.356 INFO:teuthology.orchestra.run.smithi187.stdout: debug mon = 20 2023-01-24T19:15:19.356 INFO:teuthology.orchestra.run.smithi187.stdout: debug paxos = 20 2023-01-24T19:15:19.356 INFO:teuthology.orchestra.run.smithi187.stdout: debug auth = 20 2023-01-24T19:15:19.356 INFO:teuthology.orchestra.run.smithi187.stdout: mon data avail warn = 5 2023-01-24T19:15:19.356 INFO:teuthology.orchestra.run.smithi187.stdout: mon mgr mkfs grace = 240 2023-01-24T19:15:19.357 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min pgs per osd = 4 2023-01-24T19:15:19.357 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd reporter subtree level = osd 2023-01-24T19:15:19.357 INFO:teuthology.orchestra.run.smithi187.stdout: mon osd prime pg temp = true 2023-01-24T19:15:19.357 INFO:teuthology.orchestra.run.smithi187.stdout: mon reweight min bytes per osd = 10 2023-01-24T19:15:19.357 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.358 INFO:teuthology.orchestra.run.smithi187.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T19:15:19.358 INFO:teuthology.orchestra.run.smithi187.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T19:15:19.358 INFO:teuthology.orchestra.run.smithi187.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T19:15:19.358 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.358 INFO:teuthology.orchestra.run.smithi187.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T19:15:19.358 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T19:15:19.359 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T19:15:19.359 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.359 INFO:teuthology.orchestra.run.smithi187.stdout: # 1m isn't quite enough 2023-01-24T19:15:19.359 INFO:teuthology.orchestra.run.smithi187.stdout: mon_down_mkfs_grace = 2m 2023-01-24T19:15:19.359 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.360 INFO:teuthology.orchestra.run.smithi187.stdout: mon_warn_on_filestore_osds = false 2023-01-24T19:15:19.360 INFO:teuthology.orchestra.run.smithi187.stdout: mon scrub interval = 300 2023-01-24T19:15:19.360 INFO:teuthology.orchestra.run.smithi187.stdout: 2023-01-24T19:15:19.360 INFO:teuthology.orchestra.run.smithi187.stdout:[client] 2023-01-24T19:15:19.360 INFO:teuthology.orchestra.run.smithi187.stdout: rgw cache enabled = true 2023-01-24T19:15:19.361 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable ops log = true 2023-01-24T19:15:19.361 INFO:teuthology.orchestra.run.smithi187.stdout: rgw enable usage log = true 2023-01-24T19:15:19.361 INFO:teuthology.orchestra.run.smithi187.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T19:15:19.361 INFO:teuthology.orchestra.run.smithi187.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T19:15:19.361 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.a] 2023-01-24T19:15:19.361 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.c] 2023-01-24T19:15:19.362 INFO:teuthology.orchestra.run.smithi187.stdout:[mon.b] 2023-01-24T19:15:19.362 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T19:15:19.362 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T19:15:19.426 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T19:15:19.426 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:19.426 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T19:15:19.445 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:19.446 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T19:15:19.502 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:15:19.503 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:19.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T19:15:19.503 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T19:15:19.579 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:19.580 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:19.637 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi187.front.sepia.ceph.com 2023-01-24T19:15:19.638 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:19.638 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T19:15:19.639 DEBUG:teuthology.orchestra.run.smithi187:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T19:15:19.676 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:19.677 DEBUG:teuthology.orchestra.run.smithi187:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:19.734 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T19:15:19.735 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T19:15:19.735 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T19:15:19.786 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-01-24T19:15:19.788 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T19:15:19.846 INFO:teuthology.orchestra.run.smithi187.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T19:15:19.849 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T19:15:19.849 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T19:15:19.850 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T19:15:19.891 INFO:teuthology.orchestra.run.smithi161.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T19:15:19.903 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-01-24T19:15:19.944 INFO:teuthology.orchestra.run.smithi187.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-01-24T19:15:19.956 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T19:15:19.957 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi187.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-01-24T19:15:19.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T19:15:19.984 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:19.985 INFO:tasks.ceph:role: osd.0 2023-01-24T19:15:19.985 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:15:19.985 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T19:15:20.059 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:20.060 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:20.060 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:20.061 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:20.061 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:20.061 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:20.061 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:20.482 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-01-24T19:15:20.483 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-01-24T19:15:20.484 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T19:15:20.663 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T19:15:20.698 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T19:15:20.766 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:20.767 INFO:tasks.ceph:role: osd.1 2023-01-24T19:15:20.767 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:15:20.768 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T19:15:20.841 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:20.841 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:20.842 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:20.842 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:20.842 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:20.842 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:20.842 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:20.843 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:20.843 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:20.843 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:21.266 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-01-24T19:15:21.268 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-01-24T19:15:21.268 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T19:15:21.374 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T19:15:21.406 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T19:15:21.473 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:21.474 INFO:tasks.ceph:role: osd.2 2023-01-24T19:15:21.474 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:15:21.474 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T19:15:21.549 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:21.549 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:21.550 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:21.550 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:21.550 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:21.550 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:21.550 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:21.551 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:21.551 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:21.551 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:21.993 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-01-24T19:15:21.995 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-01-24T19:15:21.995 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T19:15:22.151 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T19:15:22.186 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-24T19:15:22.253 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:22.253 INFO:tasks.ceph:role: osd.3 2023-01-24T19:15:22.254 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:15:22.254 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T19:15:22.329 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:22.330 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:22.330 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:22.330 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:22.330 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:22.331 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:22.331 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:22.331 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:22.331 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:22.331 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:22.772 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-01-24T19:15:22.773 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-01-24T19:15:22.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-24T19:15:22.946 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-24T19:15:22.979 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:23.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:23.080+0000 7fb593d6c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-24T19:15:23.082 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:23.080+0000 7fb593d6c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T19:15:23.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:23.080+0000 7fb593d6c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T19:15:23.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:23.080+0000 7fb593d6c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T19:15:23.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:23.080+0000 7fb593d6c540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T19:15:23.083 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:23.081+0000 7fb593d6c540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-24T19:15:25.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T19:15:25.952 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:26.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:26.035+0000 7f44f1a8e540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-24T19:15:26.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:26.035+0000 7f44f1a8e540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T19:15:26.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:26.035+0000 7f44f1a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T19:15:26.038 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:26.035+0000 7f44f1a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T19:15:26.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:26.035+0000 7f44f1a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T19:15:26.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:26.036+0000 7f44f1a8e540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-24T19:15:28.873 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T19:15:28.904 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:28.991 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:28.989+0000 7f6507438540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-24T19:15:28.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:28.989+0000 7f6507438540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T19:15:28.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:28.989+0000 7f6507438540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T19:15:28.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:28.989+0000 7f6507438540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T19:15:28.992 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:28.989+0000 7f6507438540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T19:15:28.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:28.990+0000 7f6507438540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-24T19:15:31.832 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T19:15:31.861 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:31.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:31.943+0000 7fd52586d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-24T19:15:31.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:31.943+0000 7fd52586d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-24T19:15:31.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:31.943+0000 7fd52586d540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T19:15:31.946 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:31.943+0000 7fd52586d540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T19:15:31.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:31.943+0000 7fd52586d540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T19:15:31.947 INFO:teuthology.orchestra.run.smithi161.stderr:2023-01-24T19:15:31.944+0000 7fd52586d540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-24T19:15:34.780 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-24T19:15:34.849 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-24T19:15:34.880 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:34.880 INFO:tasks.ceph:role: osd.4 2023-01-24T19:15:34.880 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com 2023-01-24T19:15:34.881 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T19:15:34.957 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:34.957 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:34.958 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:34.958 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:34.958 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:34.958 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:34.958 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:34.959 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:34.959 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:34.959 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:35.382 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-01-24T19:15:35.384 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-01-24T19:15:35.384 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-01-24T19:15:35.561 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-01-24T19:15:35.598 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-24T19:15:35.667 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:35.668 INFO:tasks.ceph:role: osd.5 2023-01-24T19:15:35.668 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com 2023-01-24T19:15:35.668 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T19:15:35.743 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:35.743 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:35.743 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:35.744 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:35.744 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:35.744 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:35.744 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:35.744 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:35.745 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:35.745 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:36.168 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-01-24T19:15:36.170 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-01-24T19:15:36.171 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-01-24T19:15:36.287 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-01-24T19:15:36.318 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-24T19:15:36.386 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:36.386 INFO:tasks.ceph:role: osd.6 2023-01-24T19:15:36.387 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com 2023-01-24T19:15:36.387 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T19:15:36.461 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:36.462 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:36.462 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:36.462 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:36.462 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:36.463 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:36.463 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:36.463 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:36.463 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:36.463 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:36.910 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-01-24T19:15:36.911 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-01-24T19:15:36.912 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-01-24T19:15:37.058 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-01-24T19:15:37.089 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-24T19:15:37.157 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-01-24T19:15:37.158 INFO:tasks.ceph:role: osd.7 2023-01-24T19:15:37.158 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com 2023-01-24T19:15:37.159 DEBUG:teuthology.orchestra.run.smithi187:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T19:15:37.232 INFO:teuthology.orchestra.run.smithi187.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T19:15:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T19:15:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T19:15:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T19:15:37.233 INFO:teuthology.orchestra.run.smithi187.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T19:15:37.233 INFO:teuthology.orchestra.run.smithi187.stdout: = sunit=0 swidth=0 blks 2023-01-24T19:15:37.234 INFO:teuthology.orchestra.run.smithi187.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T19:15:37.234 INFO:teuthology.orchestra.run.smithi187.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T19:15:37.234 INFO:teuthology.orchestra.run.smithi187.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T19:15:37.234 INFO:teuthology.orchestra.run.smithi187.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T19:15:37.671 INFO:teuthology.orchestra.run.smithi187.stdout:Discarding blocks...Done. 2023-01-24T19:15:37.672 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi187.front.sepia.ceph.com -o noatime 2023-01-24T19:15:37.673 DEBUG:teuthology.orchestra.run.smithi187:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-01-24T19:15:37.845 DEBUG:teuthology.orchestra.run.smithi187:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-01-24T19:15:37.882 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:37.977 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:37.975+0000 7fabe44c4540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-24T19:15:37.977 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:37.975+0000 7fabe44c4540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-24T19:15:37.977 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:37.975+0000 7fabe44c4540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T19:15:37.977 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:37.975+0000 7fabe44c4540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T19:15:37.978 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:37.975+0000 7fabe44c4540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T19:15:37.978 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:37.976+0000 7fabe44c4540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-24T19:15:40.821 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-24T19:15:40.850 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:40.934 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:40.932+0000 7f6d7d261540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-24T19:15:40.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:40.932+0000 7f6d7d261540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-24T19:15:40.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:40.932+0000 7f6d7d261540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T19:15:40.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:40.932+0000 7f6d7d261540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T19:15:40.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:40.932+0000 7f6d7d261540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T19:15:40.936 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:40.933+0000 7f6d7d261540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-24T19:15:43.781 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-24T19:15:43.851 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:43.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:43.932+0000 7ff1f5afb540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-24T19:15:43.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:43.933+0000 7ff1f5afb540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-24T19:15:43.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:43.933+0000 7ff1f5afb540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T19:15:43.935 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:43.933+0000 7ff1f5afb540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T19:15:43.936 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:43.933+0000 7ff1f5afb540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T19:15:43.936 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:43.933+0000 7ff1f5afb540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-24T19:15:46.780 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-24T19:15:46.811 DEBUG:teuthology.orchestra.run.smithi187:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:46.897 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:46.895+0000 7f7b4725f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-24T19:15:46.897 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:46.895+0000 7f7b4725f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-24T19:15:46.897 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:46.895+0000 7f7b4725f540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T19:15:46.897 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:46.895+0000 7f7b4725f540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T19:15:46.898 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:46.895+0000 7f7b4725f540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T19:15:46.898 INFO:teuthology.orchestra.run.smithi187.stderr:2023-01-24T19:15:46.896+0000 7f7b4725f540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-24T19:15:49.737 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-24T19:15:49.769 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T19:15:49.770 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:49.770 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-01-24T19:15:49.802 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:49.803 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T19:15:49.871 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:49.872 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T19:15:49.939 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:49.939 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T19:15:50.007 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:50.007 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-24T19:15:50.075 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:50.075 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T19:15:50.108 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:50.108 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-24T19:15:50.180 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:50.180 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-24T19:15:50.250 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:50.251 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-24T19:15:50.319 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:50.319 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-24T19:15:50.388 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:50.389 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T19:15:50.408 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:50.408 DEBUG:teuthology.orchestra.run.smithi187:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-01-24T19:15:50.447 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T19:15:50.447 DEBUG:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T19:15:50.450 DEBUG:teuthology.orchestra.run.smithi187:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T19:15:50.490 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2023-01-24T19:15:50.491 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBHLtBj2hTbLhAABAddOkxt1fST88coZpNO2Q== 2023-01-24T19:15:50.491 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2023-01-24T19:15:50.491 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBLLtBjG3jgBBAAuK0dgJjmvKg/nvY8psLhhA== 2023-01-24T19:15:50.491 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2023-01-24T19:15:50.491 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBOLtBjtOY3AhAAWEfHbrHX6haZ+8kLerrMlw== 2023-01-24T19:15:50.492 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2023-01-24T19:15:50.492 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBQLtBjoNgOOxAAFVZvlDDUXye09ei2uBfDTA== 2023-01-24T19:15:50.492 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2023-01-24T19:15:50.492 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBTLtBjR+JVOBAAfx+K0WEZ1xhazeFSBsX6GA== 2023-01-24T19:15:50.493 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2023-01-24T19:15:50.493 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBHLtBjtvFxMhAA5ku31gzFs5J4JkvLhb5j7Q== 2023-01-24T19:15:50.493 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2023-01-24T19:15:50.493 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBZLtBjxgE2OhAAWbTcxNpIdJTx5EBQ1kEMyQ== 2023-01-24T19:15:50.493 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2023-01-24T19:15:50.494 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBcLtBjZeupNxAAOkKAkD+AoUL7J7O9YTEkdQ== 2023-01-24T19:15:50.494 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2023-01-24T19:15:50.494 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBfLtBj24uvNxAADgc5NlgS0BcKuw/hZ0PD3A== 2023-01-24T19:15:50.494 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2023-01-24T19:15:50.494 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBiLtBjWR9tNRAADSfN1ykP/Ku5u1DzVQjn9A== 2023-01-24T19:15:50.495 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2023-01-24T19:15:50.495 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBHLtBjAl4aNRAABqT8FVZEViaUn3QTGrLW6Q== 2023-01-24T19:15:50.495 INFO:teuthology.orchestra.run.smithi161.stdout:[client.1] 2023-01-24T19:15:50.495 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQBHLtBj+U5GOBAAyqGXlDrUChxCgD1wjin36g== 2023-01-24T19:15:50.513 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.y] 2023-01-24T19:15:50.514 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBHLtBj2hTbLhAABAddOkxt1fST88coZpNO2Q== 2023-01-24T19:15:50.514 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.0] 2023-01-24T19:15:50.514 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBLLtBjG3jgBBAAuK0dgJjmvKg/nvY8psLhhA== 2023-01-24T19:15:50.514 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.1] 2023-01-24T19:15:50.515 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBOLtBjtOY3AhAAWEfHbrHX6haZ+8kLerrMlw== 2023-01-24T19:15:50.515 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.2] 2023-01-24T19:15:50.515 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBQLtBjoNgOOxAAFVZvlDDUXye09ei2uBfDTA== 2023-01-24T19:15:50.515 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.3] 2023-01-24T19:15:50.515 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBTLtBjR+JVOBAAfx+K0WEZ1xhazeFSBsX6GA== 2023-01-24T19:15:50.516 INFO:teuthology.orchestra.run.smithi187.stdout:[mgr.x] 2023-01-24T19:15:50.516 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBHLtBjtvFxMhAA5ku31gzFs5J4JkvLhb5j7Q== 2023-01-24T19:15:50.516 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.4] 2023-01-24T19:15:50.516 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBZLtBjxgE2OhAAWbTcxNpIdJTx5EBQ1kEMyQ== 2023-01-24T19:15:50.516 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.5] 2023-01-24T19:15:50.517 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBcLtBjZeupNxAAOkKAkD+AoUL7J7O9YTEkdQ== 2023-01-24T19:15:50.517 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.6] 2023-01-24T19:15:50.517 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBfLtBj24uvNxAADgc5NlgS0BcKuw/hZ0PD3A== 2023-01-24T19:15:50.517 INFO:teuthology.orchestra.run.smithi187.stdout:[osd.7] 2023-01-24T19:15:50.517 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBiLtBjWR9tNRAADSfN1ykP/Ku5u1DzVQjn9A== 2023-01-24T19:15:50.517 INFO:teuthology.orchestra.run.smithi187.stdout:[client.0] 2023-01-24T19:15:50.518 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBHLtBjAl4aNRAABqT8FVZEViaUn3QTGrLW6Q== 2023-01-24T19:15:50.518 INFO:teuthology.orchestra.run.smithi187.stdout:[client.1] 2023-01-24T19:15:50.518 INFO:teuthology.orchestra.run.smithi187.stdout: key = AQBHLtBj+U5GOBAAyqGXlDrUChxCgD1wjin36g== 2023-01-24T19:15:50.519 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:15:50.539 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:15:50.609 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.611 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.661 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.663 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.713 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.716 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.766 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.768 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.818 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:15:50.821 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T19:15:50.871 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.873 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.923 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.926 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.975 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:50.977 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:51.027 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:51.029 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T19:15:51.079 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:15:51.081 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:15:51.133 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:15:51.136 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T19:15:51.188 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T19:15:51.188 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T19:15:51.216 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T19:15:51.310 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T19:15:51.337 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-01-24T19:15:51.404 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T19:15:51.498 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-01-24T19:15:51.526 DEBUG:teuthology.orchestra.run.smithi187:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-01-24T19:15:51.555 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T19:15:51.651 DEBUG:teuthology.orchestra.run.smithi187:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-01-24T19:15:51.678 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:51.681 DEBUG:teuthology.orchestra.run.smithi187:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T19:15:51.736 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T19:15:51.737 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T19:15:51.737 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T19:15:51.739 INFO:tasks.ceph.mon.a:Started 2023-01-24T19:15:51.740 INFO:tasks.ceph.mon.c:Restarting daemon 2023-01-24T19:15:51.740 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-01-24T19:15:51.742 INFO:tasks.ceph.mon.c:Started 2023-01-24T19:15:51.742 INFO:tasks.ceph.mon.b:Restarting daemon 2023-01-24T19:15:51.743 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-01-24T19:15:51.778 INFO:tasks.ceph.mon.b:Started 2023-01-24T19:15:51.779 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T19:15:51.779 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-01-24T19:15:51.779 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-01-24T19:15:51.783 INFO:tasks.ceph.mgr.y:Started 2023-01-24T19:15:51.784 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T19:15:51.784 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T19:15:51.786 INFO:tasks.ceph.mgr.x:Started 2023-01-24T19:15:51.786 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T19:15:51.787 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T19:15:58.001 INFO:teuthology.orchestra.run.smithi161.stderr:adjusted tunables profile to default 2023-01-24T19:15:58.011 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T19:15:58.012 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:58.012 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T19:15:58.040 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:58.040 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T19:15:58.108 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:58.108 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T19:15:58.175 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-01-24T19:15:58.175 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-24T19:15:58.241 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:58.241 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-24T19:15:58.269 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:58.269 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-24T19:15:58.337 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:58.338 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-24T19:15:58.406 DEBUG:teuthology.orchestra.run.smithi187:> set -ex 2023-01-24T19:15:58.406 DEBUG:teuthology.orchestra.run.smithi187:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-24T19:15:58.475 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 689482ff-3b80-4c19-a1e0-3c72166cc073 0 2023-01-24T19:15:58.731 INFO:teuthology.orchestra.run.smithi187.stdout:0 2023-01-24T19:15:58.744 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 4d02fc80-a894-458c-be90-aca476042a66 1 2023-01-24T19:15:58.803 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:15:58.801+0000 7eff7f3fc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T19:15:58.806 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:15:58.806+0000 7fca0d6f7180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T19:15:58.875 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:15:58.874+0000 7fca0d6f7180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T19:15:58.876 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:15:58.874+0000 7eff7f3fc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T19:15:58.959 INFO:teuthology.orchestra.run.smithi187.stdout:1 2023-01-24T19:15:58.962 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:15:58.961+0000 7fca0d6f7180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T19:15:58.969 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:15:58.967+0000 7eff7f3fc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T19:15:58.971 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 49f09f72-7380-42b7-bffb-8e28edf2ca31 2 2023-01-24T19:15:59.148 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:15:59.148+0000 7fca0d6f7180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T19:15:59.167 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:15:59.165+0000 7eff7f3fc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T19:15:59.192 INFO:teuthology.orchestra.run.smithi187.stdout:2 2023-01-24T19:15:59.205 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 8905bc54-f750-4d44-9dbd-c6d5c2dfa5f6 3 2023-01-24T19:15:59.419 INFO:teuthology.orchestra.run.smithi187.stdout:3 2023-01-24T19:15:59.431 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 91234a9b-56f8-432c-a0c9-64d29ab42b02 4 2023-01-24T19:15:59.648 INFO:teuthology.orchestra.run.smithi187.stdout:4 2023-01-24T19:15:59.661 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new e7234d31-c842-4027-94c4-3244ed013b6c 5 2023-01-24T19:15:59.913 INFO:teuthology.orchestra.run.smithi187.stdout:5 2023-01-24T19:15:59.925 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new d1bfe29c-58a4-4274-9479-8ed172736b4d 6 2023-01-24T19:15:59.956 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:15:59.956+0000 7fca0d6f7180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T19:15:59.981 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:15:59.979+0000 7eff7f3fc180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T19:16:00.095 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.094+0000 7fca0d6f7180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T19:16:00.126 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.124+0000 7eff7f3fc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T19:16:00.137 INFO:teuthology.orchestra.run.smithi187.stdout:6 2023-01-24T19:16:00.150 DEBUG:teuthology.orchestra.run.smithi187:> sudo ceph --cluster ceph osd new 62821dbe-b03f-46b2-9c1e-ab2ecb698e36 7 2023-01-24T19:16:00.319 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.318+0000 7fca0d6f7180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T19:16:00.361 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.360+0000 7eff7f3fc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T19:16:00.369 INFO:teuthology.orchestra.run.smithi187.stdout:7 2023-01-24T19:16:00.382 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T19:16:00.383 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T19:16:00.385 INFO:tasks.ceph.osd.0:Started 2023-01-24T19:16:00.385 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T19:16:00.386 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T19:16:00.388 INFO:tasks.ceph.osd.1:Started 2023-01-24T19:16:00.388 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T19:16:00.389 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T19:16:00.391 INFO:tasks.ceph.osd.2:Started 2023-01-24T19:16:00.391 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-24T19:16:00.392 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-24T19:16:00.393 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.392+0000 7fca0d6f7180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T19:16:00.394 INFO:tasks.ceph.osd.3:Started 2023-01-24T19:16:00.394 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-24T19:16:00.395 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-24T19:16:00.397 INFO:tasks.ceph.osd.4:Started 2023-01-24T19:16:00.397 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-24T19:16:00.398 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-24T19:16:00.399 INFO:tasks.ceph.osd.5:Started 2023-01-24T19:16:00.400 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-24T19:16:00.400 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-24T19:16:00.402 INFO:tasks.ceph.osd.6:Started 2023-01-24T19:16:00.402 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-24T19:16:00.403 DEBUG:teuthology.orchestra.run.smithi187:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-24T19:16:00.405 INFO:tasks.ceph.osd.7:Started 2023-01-24T19:16:00.405 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T19:16:00.447 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.445+0000 7eff7f3fc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T19:16:00.571 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.570+0000 7fca0d6f7180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:16:00.593 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.591+0000 7eff7f3fc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:16:00.631 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:16:00.631 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":10,"fsid":"2fd00fc0-0054-482e-9800-3bb42aed5401","created":"2023-01-24T19:15:57.079862+0000","modified":"2023-01-24T19:16:00.364432+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T19:16:00.364432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"689482ff-3b80-4c19-a1e0-3c72166cc073","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d02fc80-a894-458c-be90-aca476042a66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49f09f72-7380-42b7-bffb-8e28edf2ca31","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8905bc54-f750-4d44-9dbd-c6d5c2dfa5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"91234a9b-56f8-432c-a0c9-64d29ab42b02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e7234d31-c842-4027-94c4-3244ed013b6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d1bfe29c-58a4-4274-9479-8ed172736b4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62821dbe-b03f-46b2-9c1e-ab2ecb698e36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:16:00.643 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T19:16:00.644 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T19:16:00.826 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.825+0000 7fca0d6f7180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T19:16:00.826 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.825+0000 7fca0d6f7180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T19:16:00.827 INFO:tasks.ceph.mgr.y.smithi161.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T19:16:00.827 INFO:tasks.ceph.mgr.y.smithi161.stderr: from .module import Module, StandbyModule 2023-01-24T19:16:00.827 INFO:tasks.ceph.mgr.y.smithi161.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T19:16:00.827 INFO:tasks.ceph.mgr.y.smithi161.stderr: from packaging.version import Version 2023-01-24T19:16:00.827 INFO:tasks.ceph.mgr.y.smithi161.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T19:16:00.828 INFO:tasks.ceph.mgr.y.smithi161.stderr: 2023-01-24T19:16:00.828 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.825+0000 7fca0d6f7180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T19:16:00.828 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:00.825+0000 7fca0d6f7180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T19:16:00.843 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.841+0000 7eff7f3fc180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T19:16:00.844 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.841+0000 7eff7f3fc180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T19:16:00.844 INFO:tasks.ceph.mgr.x.smithi187.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T19:16:00.844 INFO:tasks.ceph.mgr.x.smithi187.stderr: from .module import Module, StandbyModule 2023-01-24T19:16:00.844 INFO:tasks.ceph.mgr.x.smithi187.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T19:16:00.844 INFO:tasks.ceph.mgr.x.smithi187.stderr: from packaging.version import Version 2023-01-24T19:16:00.845 INFO:tasks.ceph.mgr.x.smithi187.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T19:16:00.845 INFO:tasks.ceph.mgr.x.smithi187.stderr: 2023-01-24T19:16:00.845 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.842+0000 7eff7f3fc180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T19:16:00.845 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:00.842+0000 7eff7f3fc180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T19:16:00.992 INFO:tasks.ceph.osd.0.smithi161.stderr:2023-01-24T19:16:00.991+0000 7fd502502540 -1 Falling back to public interface 2023-01-24T19:16:01.007 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-01-24T19:16:01.006+0000 7fc39124a540 -1 Falling back to public interface 2023-01-24T19:16:01.010 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:01.009+0000 7f852c975540 -1 Falling back to public interface 2023-01-24T19:16:01.011 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-01-24T19:16:01.009+0000 7f2c77a31540 -1 Falling back to public interface 2023-01-24T19:16:01.011 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:01.010+0000 7fca0d6f7180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T19:16:01.015 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:01.013+0000 7fa7a7bc2540 -1 Falling back to public interface 2023-01-24T19:16:01.016 INFO:tasks.ceph.osd.3.smithi161.stderr:2023-01-24T19:16:01.015+0000 7f593ceda540 -1 Falling back to public interface 2023-01-24T19:16:01.021 INFO:tasks.ceph.osd.1.smithi161.stderr:2023-01-24T19:16:01.020+0000 7fee3a81d540 -1 Falling back to public interface 2023-01-24T19:16:01.025 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:01.023+0000 7f9824ef7540 -1 Falling back to public interface 2023-01-24T19:16:01.031 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:01.030+0000 7eff7f3fc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T19:16:01.152 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:01.152+0000 7fca0d6f7180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T19:16:01.170 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:01.169+0000 7eff7f3fc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T19:16:01.269 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:01.268+0000 7fca0d6f7180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T19:16:01.285 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:01.283+0000 7eff7f3fc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T19:16:01.411 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:01.410+0000 7fca0d6f7180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:16:01.425 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:01.423+0000 7eff7f3fc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T19:16:01.450 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T19:16:01.490 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:01.490+0000 7fca0d6f7180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T19:16:01.502 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:01.500+0000 7eff7f3fc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T19:16:01.640 INFO:teuthology.misc.health.smithi161.stdout: 2023-01-24T19:16:01.640 INFO:teuthology.misc.health.smithi161.stdout:{"epoch":10,"fsid":"2fd00fc0-0054-482e-9800-3bb42aed5401","created":"2023-01-24T19:15:57.079862+0000","modified":"2023-01-24T19:16:00.364432+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T19:16:00.364432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"689482ff-3b80-4c19-a1e0-3c72166cc073","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4d02fc80-a894-458c-be90-aca476042a66","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"49f09f72-7380-42b7-bffb-8e28edf2ca31","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8905bc54-f750-4d44-9dbd-c6d5c2dfa5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"91234a9b-56f8-432c-a0c9-64d29ab42b02","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e7234d31-c842-4027-94c4-3244ed013b6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d1bfe29c-58a4-4274-9479-8ed172736b4d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"62821dbe-b03f-46b2-9c1e-ab2ecb698e36","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:16:01.651 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-01-24T19:16:01.719 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:01.719+0000 7fca0d6f7180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T19:16:01.729 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:01.727+0000 7eff7f3fc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T19:16:01.797 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:01.796+0000 7fca0d6f7180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T19:16:01.806 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:01.805+0000 7eff7f3fc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T19:16:02.407 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:02.407+0000 7fca0d6f7180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T19:16:02.413 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:02.412+0000 7eff7f3fc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T19:16:02.488 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:02.487+0000 7fca0d6f7180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T19:16:02.495 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:02.493+0000 7eff7f3fc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T19:16:02.747 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:02.745+0000 7eff7f3fc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T19:16:02.754 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:02.753+0000 7fca0d6f7180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T19:16:02.828 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:02.826+0000 7eff7f3fc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T19:16:02.834 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:02.833+0000 7fca0d6f7180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T19:16:03.009 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:03.008+0000 7eff7f3fc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T19:16:03.017 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:03.016+0000 7fca0d6f7180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T19:16:03.081 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:03.080+0000 7eff7f3fc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T19:16:03.088 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:03.087+0000 7fca0d6f7180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T19:16:03.231 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:03.230+0000 7eff7f3fc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T19:16:03.232 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:03.230+0000 7eff7f3fc180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T19:16:03.240 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:03.240+0000 7fca0d6f7180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T19:16:03.241 INFO:tasks.ceph.mgr.y.smithi161.stderr:2023-01-24T19:16:03.240+0000 7fca0d6f7180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T19:16:04.258 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:04.256+0000 7eff3be27700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-01-24T19:16:04.259 INFO:tasks.ceph.mgr.x.smithi187.stderr:2023-01-24T19:16:04.256+0000 7eff3be27700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-01-24T19:16:04.506 INFO:tasks.ceph.osd.0.smithi161.stderr:2023-01-24T19:16:04.505+0000 7fd502502540 -1 osd.0 0 log_to_monitors true 2023-01-24T19:16:04.508 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:04.508+0000 7f852c975540 -1 osd.2 0 log_to_monitors true 2023-01-24T19:16:04.518 INFO:tasks.ceph.osd.1.smithi161.stderr:2023-01-24T19:16:04.518+0000 7fee3a81d540 -1 osd.1 0 log_to_monitors true 2023-01-24T19:16:04.527 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:04.525+0000 7fa7a7bc2540 -1 osd.5 0 log_to_monitors true 2023-01-24T19:16:04.535 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-01-24T19:16:04.534+0000 7fc39124a540 -1 osd.4 0 log_to_monitors true 2023-01-24T19:16:04.546 INFO:tasks.ceph.osd.3.smithi161.stderr:2023-01-24T19:16:04.546+0000 7f593ceda540 -1 osd.3 0 log_to_monitors true 2023-01-24T19:16:04.553 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-01-24T19:16:04.551+0000 7f2c77a31540 -1 osd.7 0 log_to_monitors true 2023-01-24T19:16:04.553 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:04.551+0000 7f9824ef7540 -1 osd.6 0 log_to_monitors true 2023-01-24T19:16:06.320 INFO:tasks.ceph.osd.1.smithi161.stderr:2023-01-24T19:16:06.319+0000 7fee31428700 -1 osd.1 0 waiting for initial osdmap 2023-01-24T19:16:06.321 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-01-24T19:16:06.320+0000 7f2c6e63c700 -1 osd.7 0 waiting for initial osdmap 2023-01-24T19:16:06.326 INFO:tasks.ceph.osd.1.smithi161.stderr:2023-01-24T19:16:06.325+0000 7fee2c222700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:06.326 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-01-24T19:16:06.324+0000 7f2c69436700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:06.328 INFO:tasks.ceph.osd.0.smithi161.stderr:2023-01-24T19:16:06.328+0000 7fd4f910d700 -1 osd.0 0 waiting for initial osdmap 2023-01-24T19:16:06.329 INFO:tasks.ceph.osd.3.smithi161.stderr:2023-01-24T19:16:06.328+0000 7f5933ae5700 -1 osd.3 0 waiting for initial osdmap 2023-01-24T19:16:06.329 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:06.328+0000 7f8523580700 -1 osd.2 0 waiting for initial osdmap 2023-01-24T19:16:06.330 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:06.327+0000 7fa79e7cd700 -1 osd.5 0 waiting for initial osdmap 2023-01-24T19:16:06.330 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:06.327+0000 7f981bb02700 -1 osd.6 0 waiting for initial osdmap 2023-01-24T19:16:06.331 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-01-24T19:16:06.327+0000 7fc387e55700 -1 osd.4 0 waiting for initial osdmap 2023-01-24T19:16:06.331 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-01-24T19:16:06.329+0000 7fc37dd5c700 -1 osd.4 0 handle_osd_map injecting map crc failure 2023-01-24T19:16:06.332 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:06.332+0000 7f851e37a700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:06.333 INFO:tasks.ceph.osd.3.smithi161.stderr:2023-01-24T19:16:06.332+0000 7f592e8df700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:06.333 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:06.331+0000 7f98168fc700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:06.334 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-01-24T19:16:06.333+0000 7fc382c4f700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:06.335 INFO:tasks.ceph.osd.0.smithi161.stderr:2023-01-24T19:16:06.334+0000 7fd4f3f07700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:06.335 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:06.333+0000 7fa7995c7700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T19:16:07.332 INFO:tasks.ceph.osd.0.smithi161.stderr:2023-01-24T19:16:07.330+0000 7fd4ef014700 -1 osd.0 12 handle_osd_map injecting map crc failure 2023-01-24T19:16:08.457 DEBUG:teuthology.orchestra.run.smithi161:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T19:16:08.767 INFO:teuthology.misc.health.smithi161.stdout: 2023-01-24T19:16:08.767 INFO:teuthology.misc.health.smithi161.stdout:{"epoch":14,"fsid":"2fd00fc0-0054-482e-9800-3bb42aed5401","created":"2023-01-24T19:15:57.079862+0000","modified":"2023-01-24T19:16:08.335710+0000","last_up_change":"2023-01-24T19:16:07.324711+0000","last_in_change":"2023-01-24T19:16:00.364432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"689482ff-3b80-4c19-a1e0-3c72166cc073","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6803","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":106361}]},"public_addr":"172.21.15.161:6800/106361","cluster_addr":"172.21.15.161:6801/106361","heartbeat_back_addr":"172.21.15.161:6803/106361","heartbeat_front_addr":"172.21.15.161:6802/106361","state":["exists","up"]},{"osd":1,"uuid":"4d02fc80-a894-458c-be90-aca476042a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":106378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":106378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":106378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":106378}]},"public_addr":"172.21.15.161:6812/106378","cluster_addr":"172.21.15.161:6813/106378","heartbeat_back_addr":"172.21.15.161:6815/106378","heartbeat_front_addr":"172.21.15.161:6814/106378","state":["exists","up"]},{"osd":2,"uuid":"49f09f72-7380-42b7-bffb-8e28edf2ca31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":106363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":106363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":106363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":106363}]},"public_addr":"172.21.15.161:6804/106363","cluster_addr":"172.21.15.161:6805/106363","heartbeat_back_addr":"172.21.15.161:6807/106363","heartbeat_front_addr":"172.21.15.161:6806/106363","state":["exists","up"]},{"osd":3,"uuid":"8905bc54-f750-4d44-9dbd-c6d5c2dfa5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6809","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":106362}]},"public_addr":"172.21.15.161:6808/106362","cluster_addr":"172.21.15.161:6809/106362","heartbeat_back_addr":"172.21.15.161:6811/106362","heartbeat_front_addr":"172.21.15.161:6810/106362","state":["exists","up"]},{"osd":4,"uuid":"91234a9b-56f8-432c-a0c9-64d29ab42b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":106337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":106337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6803","nonce":106337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":106337}]},"public_addr":"172.21.15.187:6800/106337","cluster_addr":"172.21.15.187:6801/106337","heartbeat_back_addr":"172.21.15.187:6803/106337","heartbeat_front_addr":"172.21.15.187:6802/106337","state":["exists","up"]},{"osd":5,"uuid":"e7234d31-c842-4027-94c4-3244ed013b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":106361}]},"public_addr":"172.21.15.187:6808/106361","cluster_addr":"172.21.15.187:6809/106361","heartbeat_back_addr":"172.21.15.187:6811/106361","heartbeat_front_addr":"172.21.15.187:6810/106361","state":["exists","up"]},{"osd":6,"uuid":"d1bfe29c-58a4-4274-9479-8ed172736b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":106362}]},"public_addr":"172.21.15.187:6812/106362","cluster_addr":"172.21.15.187:6813/106362","heartbeat_back_addr":"172.21.15.187:6815/106362","heartbeat_front_addr":"172.21.15.187:6814/106362","state":["exists","up"]},{"osd":7,"uuid":"62821dbe-b03f-46b2-9c1e-ab2ecb698e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":106338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":106338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6807","nonce":106338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":106338}]},"public_addr":"172.21.15.187:6804/106338","cluster_addr":"172.21.15.187:6805/106338","heartbeat_back_addr":"172.21.15.187:6807/106338","heartbeat_front_addr":"172.21.15.187:6806/106338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.478564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.494881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.547970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.559120+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.569592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.487415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.540582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.595248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:16:08.780 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-01-24T19:16:08.780 INFO:tasks.ceph:Creating RBD pool 2023-01-24T19:16:08.781 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T19:16:09.131 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:09.130+0000 7fa7926d0700 -1 osd.5 13 handle_osd_map injecting map crc failure 2023-01-24T19:16:09.351 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'rbd' created 2023-01-24T19:16:09.362 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T19:16:09.416 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-01-24T19:16:09.413+0000 7f2c64543700 -1 osd.7 14 handle_osd_map injecting map crc failure 2023-01-24T19:16:09.613 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:09.611+0000 7f8519487700 -1 osd.2 13 handle_osd_map injecting map crc failure 2023-01-24T19:16:10.354 INFO:teuthology.orchestra.run.smithi161.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T19:16:10.366 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T19:16:10.367 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T19:16:10.368 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T19:16:10.368 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T19:16:10.369 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T19:16:10.757 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:16:10.769 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6816","nonce":105674}]},"active_addr":"172.21.15.187:6816/105674","active_change":"2023-01-24T19:16:03.245218+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4109,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":300437422}]},{"addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":55213402}]},{"addrvec":[{"type":"v2","addr":"172.21.15.187:0","nonce":3714838196}]}]}} 2023-01-24T19:16:10.773 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T19:16:10.773 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T19:16:10.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T19:16:11.083 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:16:11.083 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":16,"fsid":"2fd00fc0-0054-482e-9800-3bb42aed5401","created":"2023-01-24T19:15:57.079862+0000","modified":"2023-01-24T19:16:10.345659+0000","last_up_change":"2023-01-24T19:16:07.324711+0000","last_in_change":"2023-01-24T19:16:00.364432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T19:16:09.101403+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-01-24T19:16:09.292367+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"689482ff-3b80-4c19-a1e0-3c72166cc073","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6803","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":106361}]},"public_addr":"172.21.15.161:6800/106361","cluster_addr":"172.21.15.161:6801/106361","heartbeat_back_addr":"172.21.15.161:6803/106361","heartbeat_front_addr":"172.21.15.161:6802/106361","state":["exists","up"]},{"osd":1,"uuid":"4d02fc80-a894-458c-be90-aca476042a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":106378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":106378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":106378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":106378}]},"public_addr":"172.21.15.161:6812/106378","cluster_addr":"172.21.15.161:6813/106378","heartbeat_back_addr":"172.21.15.161:6815/106378","heartbeat_front_addr":"172.21.15.161:6814/106378","state":["exists","up"]},{"osd":2,"uuid":"49f09f72-7380-42b7-bffb-8e28edf2ca31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":106363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":106363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":106363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":106363}]},"public_addr":"172.21.15.161:6804/106363","cluster_addr":"172.21.15.161:6805/106363","heartbeat_back_addr":"172.21.15.161:6807/106363","heartbeat_front_addr":"172.21.15.161:6806/106363","state":["exists","up"]},{"osd":3,"uuid":"8905bc54-f750-4d44-9dbd-c6d5c2dfa5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6809","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":106362}]},"public_addr":"172.21.15.161:6808/106362","cluster_addr":"172.21.15.161:6809/106362","heartbeat_back_addr":"172.21.15.161:6811/106362","heartbeat_front_addr":"172.21.15.161:6810/106362","state":["exists","up"]},{"osd":4,"uuid":"91234a9b-56f8-432c-a0c9-64d29ab42b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":106337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":106337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6803","nonce":106337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":106337}]},"public_addr":"172.21.15.187:6800/106337","cluster_addr":"172.21.15.187:6801/106337","heartbeat_back_addr":"172.21.15.187:6803/106337","heartbeat_front_addr":"172.21.15.187:6802/106337","state":["exists","up"]},{"osd":5,"uuid":"e7234d31-c842-4027-94c4-3244ed013b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":106361}]},"public_addr":"172.21.15.187:6808/106361","cluster_addr":"172.21.15.187:6809/106361","heartbeat_back_addr":"172.21.15.187:6811/106361","heartbeat_front_addr":"172.21.15.187:6810/106361","state":["exists","up"]},{"osd":6,"uuid":"d1bfe29c-58a4-4274-9479-8ed172736b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":106362}]},"public_addr":"172.21.15.187:6812/106362","cluster_addr":"172.21.15.187:6813/106362","heartbeat_back_addr":"172.21.15.187:6815/106362","heartbeat_front_addr":"172.21.15.187:6814/106362","state":["exists","up"]},{"osd":7,"uuid":"62821dbe-b03f-46b2-9c1e-ab2ecb698e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":106338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":106338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6807","nonce":106338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":106338}]},"public_addr":"172.21.15.187:6804/106338","cluster_addr":"172.21.15.187:6805/106338","heartbeat_back_addr":"172.21.15.187:6807/106338","heartbeat_front_addr":"172.21.15.187:6806/106338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.478564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.494881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.547970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.559120+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.569592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.487415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.540582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.595248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:16:11.098 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T19:16:11.099 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T19:16:11.359 INFO:tasks.ceph.osd.1.smithi161.stderr:2023-01-24T19:16:11.357+0000 7fee2732f700 -1 osd.1 16 handle_osd_map injecting map crc failure 2023-01-24T19:16:11.422 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:16:11.423 INFO:teuthology.orchestra.run.smithi161.stdout:{"epoch":17,"fsid":"2fd00fc0-0054-482e-9800-3bb42aed5401","created":"2023-01-24T19:15:57.079862+0000","modified":"2023-01-24T19:16:11.351808+0000","last_up_change":"2023-01-24T19:16:07.324711+0000","last_in_change":"2023-01-24T19:16:00.364432+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T19:16:09.101403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2023-01-24T19:16:09.292367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"689482ff-3b80-4c19-a1e0-3c72166cc073","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6801","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6803","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":106361}]},"public_addr":"172.21.15.161:6800/106361","cluster_addr":"172.21.15.161:6801/106361","heartbeat_back_addr":"172.21.15.161:6803/106361","heartbeat_front_addr":"172.21.15.161:6802/106361","state":["exists","up"]},{"osd":1,"uuid":"4d02fc80-a894-458c-be90-aca476042a66","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":106378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6813","nonce":106378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6815","nonce":106378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":106378}]},"public_addr":"172.21.15.161:6812/106378","cluster_addr":"172.21.15.161:6813/106378","heartbeat_back_addr":"172.21.15.161:6815/106378","heartbeat_front_addr":"172.21.15.161:6814/106378","state":["exists","up"]},{"osd":2,"uuid":"49f09f72-7380-42b7-bffb-8e28edf2ca31","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":106363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6805","nonce":106363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6807","nonce":106363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":106363}]},"public_addr":"172.21.15.161:6804/106363","cluster_addr":"172.21.15.161:6805/106363","heartbeat_back_addr":"172.21.15.161:6807/106363","heartbeat_front_addr":"172.21.15.161:6806/106363","state":["exists","up"]},{"osd":3,"uuid":"8905bc54-f750-4d44-9dbd-c6d5c2dfa5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6809","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6811","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":106362}]},"public_addr":"172.21.15.161:6808/106362","cluster_addr":"172.21.15.161:6809/106362","heartbeat_back_addr":"172.21.15.161:6811/106362","heartbeat_front_addr":"172.21.15.161:6810/106362","state":["exists","up"]},{"osd":4,"uuid":"91234a9b-56f8-432c-a0c9-64d29ab42b02","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6800","nonce":106337}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6801","nonce":106337}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6803","nonce":106337}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6802","nonce":106337}]},"public_addr":"172.21.15.187:6800/106337","cluster_addr":"172.21.15.187:6801/106337","heartbeat_back_addr":"172.21.15.187:6803/106337","heartbeat_front_addr":"172.21.15.187:6802/106337","state":["exists","up"]},{"osd":5,"uuid":"e7234d31-c842-4027-94c4-3244ed013b6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6808","nonce":106361}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6809","nonce":106361}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6811","nonce":106361}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6810","nonce":106361}]},"public_addr":"172.21.15.187:6808/106361","cluster_addr":"172.21.15.187:6809/106361","heartbeat_back_addr":"172.21.15.187:6811/106361","heartbeat_front_addr":"172.21.15.187:6810/106361","state":["exists","up"]},{"osd":6,"uuid":"d1bfe29c-58a4-4274-9479-8ed172736b4d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6812","nonce":106362}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6813","nonce":106362}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6815","nonce":106362}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6814","nonce":106362}]},"public_addr":"172.21.15.187:6812/106362","cluster_addr":"172.21.15.187:6813/106362","heartbeat_back_addr":"172.21.15.187:6815/106362","heartbeat_front_addr":"172.21.15.187:6814/106362","state":["exists","up"]},{"osd":7,"uuid":"62821dbe-b03f-46b2-9c1e-ab2ecb698e36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6804","nonce":106338}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6805","nonce":106338}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6807","nonce":106338}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.187:6806","nonce":106338}]},"public_addr":"172.21.15.187:6804/106338","cluster_addr":"172.21.15.187:6805/106338","heartbeat_back_addr":"172.21.15.187:6807/106338","heartbeat_front_addr":"172.21.15.187:6806/106338","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.478564+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.494881+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.547970+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.559120+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.569592+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.487415+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.540582+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T19:16:05.595248+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T19:16:11.436 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T19:16:11.437 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T19:16:11.437 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T19:16:11.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-24T19:16:11.438 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-24T19:16:11.439 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-24T19:16:11.440 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-24T19:16:11.441 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-24T19:16:11.715 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:11.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T19:16:11.722 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:11.722 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T19:16:11.732 INFO:teuthology.orchestra.run.smithi161.stdout:55834574852 2023-01-24T19:16:11.733 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T19:16:11.735 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:11.736 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T19:16:11.737 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:11.737 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T19:16:11.739 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:11.740 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T19:16:11.743 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:11.743 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T19:16:11.754 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:11.754 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T19:16:12.323 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:12.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2023-01-24T19:16:12.352 INFO:teuthology.orchestra.run.smithi161.stdout:55834574849 2023-01-24T19:16:12.356 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:12.362 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:12.363 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:12.368 INFO:teuthology.orchestra.run.smithi161.stdout:55834574849 2023-01-24T19:16:12.376 INFO:teuthology.orchestra.run.smithi161.stdout:55834574849 2023-01-24T19:16:12.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2023-01-24T19:16:12.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2023-01-24T19:16:12.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.5 2023-01-24T19:16:12.397 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:12.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2023-01-24T19:16:12.399 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.0 2023-01-24T19:16:12.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2023-01-24T19:16:12.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2023-01-24T19:16:13.345 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T19:16:13.388 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T19:16:13.389 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T19:16:13.396 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T19:16:13.399 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T19:16:13.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T19:16:13.400 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T19:16:13.413 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T19:16:13.815 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:13.812+0000 7f8517483700 -1 osd.2 16 handle_osd_map injecting map crc failure 2023-01-24T19:16:13.942 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:13.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2023-01-24T19:16:13.965 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:13.968 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:13.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2023-01-24T19:16:13.983 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:14.002 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:14.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.2 2023-01-24T19:16:14.022 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:14.029 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:14.036 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:14.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2023-01-24T19:16:14.045 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:14.047 INFO:teuthology.orchestra.run.smithi161.stdout:55834574852 2023-01-24T19:16:14.050 INFO:teuthology.orchestra.run.smithi161.stdout:55834574850 2023-01-24T19:16:14.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.0 2023-01-24T19:16:14.059 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:14.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.5 2023-01-24T19:16:14.064 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:14.065 INFO:teuthology.orchestra.run.smithi161.stdout:55834574851 2023-01-24T19:16:14.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.1 2023-01-24T19:16:14.066 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:14.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2023-01-24T19:16:14.080 DEBUG:teuthology.parallel:result is None 2023-01-24T19:16:14.080 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T19:16:14.080 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T19:16:14.425 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:16:14.427 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-01-24T19:16:14.438 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T19:16:13.250570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41336,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":754933384,"statfs":{"total":773094113280,"available":773051785216,"internally_reserved":0,"allocated":1179648,"data_stored":1386971,"data_compressed":9602,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.352349+0000","last_change":"2023-01-24T19:16:10.352349+0000","last_active":"2023-01-24T19:16:10.352349+0000","last_peered":"2023-01-24T19:16:10.352349+0000","last_clean":"2023-01-24T19:16:10.352349+0000","last_became_active":"2023-01-24T19:16:10.352020+0000","last_became_peered":"2023-01-24T19:16:10.352020+0000","last_unstale":"2023-01-24T19:16:10.352349+0000","last_undegraded":"2023-01-24T19:16:10.352349+0000","last_fullsized":"2023-01-24T19:16:10.352349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:25:28.409248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560672+0000","last_change":"2023-01-24T19:16:10.560672+0000","last_active":"2023-01-24T19:16:10.560672+0000","last_peered":"2023-01-24T19:16:10.560672+0000","last_clean":"2023-01-24T19:16:10.560672+0000","last_became_active":"2023-01-24T19:16:10.560425+0000","last_became_peered":"2023-01-24T19:16:10.560425+0000","last_unstale":"2023-01-24T19:16:10.560672+0000","last_undegraded":"2023-01-24T19:16:10.560672+0000","last_fullsized":"2023-01-24T19:16:10.560672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T07:13:53.677151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560720+0000","last_change":"2023-01-24T19:16:10.560720+0000","last_active":"2023-01-24T19:16:10.560720+0000","last_peered":"2023-01-24T19:16:10.560720+0000","last_clean":"2023-01-24T19:16:10.560720+0000","last_became_active":"2023-01-24T19:16:10.560466+0000","last_became_peered":"2023-01-24T19:16:10.560466+0000","last_unstale":"2023-01-24T19:16:10.560720+0000","last_undegraded":"2023-01-24T19:16:10.560720+0000","last_fullsized":"2023-01-24T19:16:10.560720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:48:52.611977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560770+0000","last_change":"2023-01-24T19:16:10.560770+0000","last_active":"2023-01-24T19:16:10.560770+0000","last_peered":"2023-01-24T19:16:10.560770+0000","last_clean":"2023-01-24T19:16:10.560770+0000","last_became_active":"2023-01-24T19:16:10.560421+0000","last_became_peered":"2023-01-24T19:16:10.560421+0000","last_unstale":"2023-01-24T19:16:10.560770+0000","last_undegraded":"2023-01-24T19:16:10.560770+0000","last_fullsized":"2023-01-24T19:16:10.560770+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:28:04.467095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560665+0000","last_change":"2023-01-24T19:16:10.560665+0000","last_active":"2023-01-24T19:16:10.560665+0000","last_peered":"2023-01-24T19:16:10.560665+0000","last_clean":"2023-01-24T19:16:10.560665+0000","last_became_active":"2023-01-24T19:16:10.560389+0000","last_became_peered":"2023-01-24T19:16:10.560389+0000","last_unstale":"2023-01-24T19:16:10.560665+0000","last_undegraded":"2023-01-24T19:16:10.560665+0000","last_fullsized":"2023-01-24T19:16:10.560665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T22:00:30.596599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":158,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:11.329206+0000","last_change":"2023-01-24T19:16:10.352513+0000","last_active":"2023-01-24T19:16:11.329206+0000","last_peered":"2023-01-24T19:16:11.329206+0000","last_clean":"2023-01-24T19:16:11.329206+0000","last_became_active":"2023-01-24T19:16:10.352247+0000","last_became_peered":"2023-01-24T19:16:10.352247+0000","last_unstale":"2023-01-24T19:16:11.329206+0000","last_undegraded":"2023-01-24T19:16:11.329206+0000","last_fullsized":"2023-01-24T19:16:11.329206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:56:54.967485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.356199+0000","last_change":"2023-01-24T19:16:10.355264+0000","last_active":"2023-01-24T19:16:10.356199+0000","last_peered":"2023-01-24T19:16:10.356199+0000","last_clean":"2023-01-24T19:16:10.356199+0000","last_became_active":"2023-01-24T19:16:10.354944+0000","last_became_peered":"2023-01-24T19:16:10.354944+0000","last_unstale":"2023-01-24T19:16:10.356199+0000","last_undegraded":"2023-01-24T19:16:10.356199+0000","last_fullsized":"2023-01-24T19:16:10.356199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T23:24:43.587721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.358164+0000","last_change":"2023-01-24T19:16:10.357705+0000","last_active":"2023-01-24T19:16:10.358164+0000","last_peered":"2023-01-24T19:16:10.358164+0000","last_clean":"2023-01-24T19:16:10.358164+0000","last_became_active":"2023-01-24T19:16:10.357396+0000","last_became_peered":"2023-01-24T19:16:10.357396+0000","last_unstale":"2023-01-24T19:16:10.358164+0000","last_undegraded":"2023-01-24T19:16:10.358164+0000","last_fullsized":"2023-01-24T19:16:10.358164+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:14:40.811605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.624044+0000","last_change":"2023-01-24T19:16:10.623623+0000","last_active":"2023-01-24T19:16:10.624044+0000","last_peered":"2023-01-24T19:16:10.624044+0000","last_clean":"2023-01-24T19:16:10.624044+0000","last_became_active":"2023-01-24T19:16:10.623356+0000","last_became_peered":"2023-01-24T19:16:10.623356+0000","last_unstale":"2023-01-24T19:16:10.624044+0000","last_undegraded":"2023-01-24T19:16:10.624044+0000","last_fullsized":"2023-01-24T19:16:10.624044+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:20:07.029510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":9602,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5596,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366244,"statfs":{"total":96636764160,"available":96631033856,"internally_reserved":0,"allocated":188416,"data_stored":568585,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":1,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5604,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366236,"statfs":{"total":96636764160,"available":96631025664,"internally_reserved":0,"allocated":196608,"data_stored":573580,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5068,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366772,"statfs":{"total":96636764160,"available":96631574528,"internally_reserved":0,"allocated":135168,"data_stored":38502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5000,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366840,"statfs":{"total":96636764160,"available":96631644160,"internally_reserved":0,"allocated":131072,"data_stored":39021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":41420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":39657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T19:16:14.439 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T19:16:14.743 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:16:14.745 INFO:teuthology.orchestra.run.smithi161.stderr:dumped all 2023-01-24T19:16:14.756 INFO:teuthology.orchestra.run.smithi161.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-24T19:16:13.250570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":41336,"kb_used_data":1152,"kb_used_omap":0,"kb_used_meta":40064,"kb_avail":754933384,"statfs":{"total":773094113280,"available":773051785216,"internally_reserved":0,"allocated":1179648,"data_stored":1386971,"data_compressed":9602,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":41025536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.352349+0000","last_change":"2023-01-24T19:16:10.352349+0000","last_active":"2023-01-24T19:16:10.352349+0000","last_peered":"2023-01-24T19:16:10.352349+0000","last_clean":"2023-01-24T19:16:10.352349+0000","last_became_active":"2023-01-24T19:16:10.352020+0000","last_became_peered":"2023-01-24T19:16:10.352020+0000","last_unstale":"2023-01-24T19:16:10.352349+0000","last_undegraded":"2023-01-24T19:16:10.352349+0000","last_fullsized":"2023-01-24T19:16:10.352349+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:25:28.409248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560672+0000","last_change":"2023-01-24T19:16:10.560672+0000","last_active":"2023-01-24T19:16:10.560672+0000","last_peered":"2023-01-24T19:16:10.560672+0000","last_clean":"2023-01-24T19:16:10.560672+0000","last_became_active":"2023-01-24T19:16:10.560425+0000","last_became_peered":"2023-01-24T19:16:10.560425+0000","last_unstale":"2023-01-24T19:16:10.560672+0000","last_undegraded":"2023-01-24T19:16:10.560672+0000","last_fullsized":"2023-01-24T19:16:10.560672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T07:13:53.677151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560720+0000","last_change":"2023-01-24T19:16:10.560720+0000","last_active":"2023-01-24T19:16:10.560720+0000","last_peered":"2023-01-24T19:16:10.560720+0000","last_clean":"2023-01-24T19:16:10.560720+0000","last_became_active":"2023-01-24T19:16:10.560466+0000","last_became_peered":"2023-01-24T19:16:10.560466+0000","last_unstale":"2023-01-24T19:16:10.560720+0000","last_undegraded":"2023-01-24T19:16:10.560720+0000","last_fullsized":"2023-01-24T19:16:10.560720+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:48:52.611977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560770+0000","last_change":"2023-01-24T19:16:10.560770+0000","last_active":"2023-01-24T19:16:10.560770+0000","last_peered":"2023-01-24T19:16:10.560770+0000","last_clean":"2023-01-24T19:16:10.560770+0000","last_became_active":"2023-01-24T19:16:10.560421+0000","last_became_peered":"2023-01-24T19:16:10.560421+0000","last_unstale":"2023-01-24T19:16:10.560770+0000","last_undegraded":"2023-01-24T19:16:10.560770+0000","last_fullsized":"2023-01-24T19:16:10.560770+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:28:04.467095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":8,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.560665+0000","last_change":"2023-01-24T19:16:10.560665+0000","last_active":"2023-01-24T19:16:10.560665+0000","last_peered":"2023-01-24T19:16:10.560665+0000","last_clean":"2023-01-24T19:16:10.560665+0000","last_became_active":"2023-01-24T19:16:10.560389+0000","last_became_peered":"2023-01-24T19:16:10.560389+0000","last_unstale":"2023-01-24T19:16:10.560665+0000","last_undegraded":"2023-01-24T19:16:10.560665+0000","last_fullsized":"2023-01-24T19:16:10.560665+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T22:00:30.596599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"16'117","reported_seq":158,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:11.329206+0000","last_change":"2023-01-24T19:16:10.352513+0000","last_active":"2023-01-24T19:16:11.329206+0000","last_peered":"2023-01-24T19:16:11.329206+0000","last_clean":"2023-01-24T19:16:11.329206+0000","last_became_active":"2023-01-24T19:16:10.352247+0000","last_became_peered":"2023-01-24T19:16:10.352247+0000","last_unstale":"2023-01-24T19:16:11.329206+0000","last_undegraded":"2023-01-24T19:16:11.329206+0000","last_fullsized":"2023-01-24T19:16:11.329206+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:56:54.967485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.356199+0000","last_change":"2023-01-24T19:16:10.355264+0000","last_active":"2023-01-24T19:16:10.356199+0000","last_peered":"2023-01-24T19:16:10.356199+0000","last_clean":"2023-01-24T19:16:10.356199+0000","last_became_active":"2023-01-24T19:16:10.354944+0000","last_became_peered":"2023-01-24T19:16:10.354944+0000","last_unstale":"2023-01-24T19:16:10.356199+0000","last_undegraded":"2023-01-24T19:16:10.356199+0000","last_fullsized":"2023-01-24T19:16:10.356199+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T23:24:43.587721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.358164+0000","last_change":"2023-01-24T19:16:10.357705+0000","last_active":"2023-01-24T19:16:10.358164+0000","last_peered":"2023-01-24T19:16:10.358164+0000","last_clean":"2023-01-24T19:16:10.358164+0000","last_became_active":"2023-01-24T19:16:10.357396+0000","last_became_peered":"2023-01-24T19:16:10.357396+0000","last_unstale":"2023-01-24T19:16:10.358164+0000","last_undegraded":"2023-01-24T19:16:10.358164+0000","last_fullsized":"2023-01-24T19:16:10.358164+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:14:40.811605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-01-24T19:16:10.624044+0000","last_change":"2023-01-24T19:16:10.623623+0000","last_active":"2023-01-24T19:16:10.624044+0000","last_peered":"2023-01-24T19:16:10.624044+0000","last_clean":"2023-01-24T19:16:10.624044+0000","last_became_active":"2023-01-24T19:16:10.623356+0000","last_became_peered":"2023-01-24T19:16:10.623356+0000","last_unstale":"2023-01-24T19:16:10.624044+0000","last_undegraded":"2023-01-24T19:16:10.624044+0000","last_fullsized":"2023-01-24T19:16:10.624044+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T19:16:09.342519+0000","last_clean_scrub_stamp":"2023-01-24T19:16:09.342519+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:20:07.029510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":9602,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5596,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366244,"statfs":{"total":96636764160,"available":96631033856,"internally_reserved":0,"allocated":188416,"data_stored":568585,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92800000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5008,"kb_used_data":136,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366832,"statfs":{"total":96636764160,"available":96631635968,"internally_reserved":0,"allocated":139264,"data_stored":40610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.431}]}]},{"osd":1,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5604,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":94366236,"statfs":{"total":96636764160,"available":96631025664,"internally_reserved":0,"allocated":196608,"data_stored":573580,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5068,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366772,"statfs":{"total":96636764160,"available":96631574528,"internally_reserved":0,"allocated":135168,"data_stored":38502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574850,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5000,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366840,"statfs":{"total":96636764160,"available":96631644160,"internally_reserved":0,"allocated":131072,"data_stored":39021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366824,"statfs":{"total":96636764160,"available":96631627776,"internally_reserved":0,"allocated":147456,"data_stored":45596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":106496,"data_stored":41420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5004,"kb_used_data":132,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366836,"statfs":{"total":96636764160,"available":96631640064,"internally_reserved":0,"allocated":135168,"data_stored":39657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":4801,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T19:16:14.757 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T19:16:14.758 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T19:16:14.758 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T19:16:14.758 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T19:16:15.135 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-01-24T19:16:15.136 INFO:teuthology.orchestra.run.smithi161.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T19:16:15.149 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T19:16:15.149 INFO:teuthology.run_tasks:Running task exec... 2023-01-24T19:16:15.160 INFO:teuthology.task.exec:Executing custom commands... 2023-01-24T19:16:15.160 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:16:15.161 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'while ! ceph balancer status ; do sleep 1 ; done' 2023-01-24T19:16:15.467 INFO:teuthology.orchestra.run.smithi161.stdout:{ 2023-01-24T19:16:15.467 INFO:teuthology.orchestra.run.smithi161.stdout: "active": true, 2023-01-24T19:16:15.467 INFO:teuthology.orchestra.run.smithi161.stdout: "last_optimize_duration": "0:00:00.000461", 2023-01-24T19:16:15.468 INFO:teuthology.orchestra.run.smithi161.stdout: "last_optimize_started": "Tue Jan 24 19:16:03 2023", 2023-01-24T19:16:15.468 INFO:teuthology.orchestra.run.smithi161.stdout: "mode": "upmap", 2023-01-24T19:16:15.468 INFO:teuthology.orchestra.run.smithi161.stdout: "no_optimization_needed": false, 2023-01-24T19:16:15.468 INFO:teuthology.orchestra.run.smithi161.stdout: "optimize_result": "No pools available", 2023-01-24T19:16:15.468 INFO:teuthology.orchestra.run.smithi161.stdout: "plans": [] 2023-01-24T19:16:15.469 INFO:teuthology.orchestra.run.smithi161.stdout:} 2023-01-24T19:16:15.481 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer mode crush-compat' 2023-01-24T19:16:16.387 INFO:tasks.ceph.osd.4.smithi187.stderr:2023-01-24T19:16:16.386+0000 7fc37dd5c700 -1 osd.4 17 handle_osd_map injecting map crc failure 2023-01-24T19:16:16.416 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph balancer on' 2023-01-24T19:16:16.732 INFO:teuthology.run_tasks:Running task exec... 2023-01-24T19:16:16.741 INFO:teuthology.task.exec:Executing custom commands... 2023-01-24T19:16:16.741 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi161.front.sepia.ceph.com 2023-01-24T19:16:16.741 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create base 4' 2023-01-24T19:16:17.844 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'base' created 2023-01-24T19:16:17.857 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool application enable base rados' 2023-01-24T19:16:17.956 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:17.954+0000 7f8517483700 -1 osd.2 18 handle_osd_map injecting map crc failure 2023-01-24T19:16:18.848 INFO:tasks.ceph.osd.3.smithi161.stderr:2023-01-24T19:16:18.846+0000 7f59299ec700 -1 osd.3 19 handle_osd_map injecting map crc failure 2023-01-24T19:16:18.852 INFO:teuthology.orchestra.run.smithi161.stderr:enabled application 'rados' on pool 'base' 2023-01-24T19:16:18.868 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2023-01-24T19:16:19.859 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:19.858+0000 7fa7946d4700 -1 osd.5 20 handle_osd_map injecting map crc failure 2023-01-24T19:16:19.869 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cache' created 2023-01-24T19:16:19.881 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add base cache' 2023-01-24T19:16:20.870 INFO:teuthology.orchestra.run.smithi161.stderr:pool 'cache' is now (or already was) a tier of 'base' 2023-01-24T19:16:20.884 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2023-01-24T19:16:21.357 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:21.356+0000 7f980fa05700 -1 osd.6 21 handle_osd_map injecting map crc failure 2023-01-24T19:16:21.660 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:21.659+0000 7fa7946d4700 -1 osd.5 21 handle_osd_map injecting map crc failure 2023-01-24T19:16:21.891 INFO:teuthology.orchestra.run.smithi161.stderr:set cache-mode for pool 'cache' to writeback 2023-01-24T19:16:21.902 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay base cache' 2023-01-24T19:16:22.887 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-01-24T19:16:22.886+0000 7f2c64543700 -1 osd.7 23 handle_osd_map injecting map crc failure 2023-01-24T19:16:22.887 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:22.886+0000 7f9811a09700 -1 osd.6 22 handle_osd_map injecting map crc failure 2023-01-24T19:16:22.891 INFO:teuthology.orchestra.run.smithi161.stderr:overlay for 'base' is now (or already was) 'cache' 2023-01-24T19:16:22.905 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2023-01-24T19:16:23.896 INFO:teuthology.orchestra.run.smithi161.stderr:set pool 4 hit_set_type to bloom 2023-01-24T19:16:23.908 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2023-01-24T19:16:24.258 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:24.256+0000 7f8517483700 -1 osd.2 24 handle_osd_map injecting map crc failure 2023-01-24T19:16:24.258 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:24.257+0000 7fa7926d0700 -1 osd.5 24 handle_osd_map injecting map crc failure 2023-01-24T19:16:24.896 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:24.896+0000 7f9811a09700 -1 osd.6 25 handle_osd_map injecting map crc failure 2023-01-24T19:16:24.900 INFO:teuthology.orchestra.run.smithi161.stderr:set pool 4 hit_set_count to 8 2023-01-24T19:16:24.913 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 3600' 2023-01-24T19:16:25.511 INFO:tasks.ceph.osd.2.smithi161.stderr:2023-01-24T19:16:25.510+0000 7f8517483700 -1 osd.2 25 handle_osd_map injecting map crc failure 2023-01-24T19:16:25.905 INFO:teuthology.orchestra.run.smithi161.stderr:set pool 4 hit_set_period to 3600 2023-01-24T19:16:25.917 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2023-01-24T19:16:26.909 INFO:tasks.ceph.osd.7.smithi187.stderr:2023-01-24T19:16:26.908+0000 7f2c64543700 -1 osd.7 27 handle_osd_map injecting map crc failure 2023-01-24T19:16:26.916 INFO:teuthology.orchestra.run.smithi161.stderr:set pool 4 target_max_objects to 250 2023-01-24T19:16:26.928 DEBUG:teuthology.orchestra.run.smithi161:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache min_read_recency_for_promote 2' 2023-01-24T19:16:27.919 INFO:teuthology.orchestra.run.smithi161.stderr:set pool 4 min_read_recency_for_promote to 2 2023-01-24T19:16:27.932 INFO:teuthology.run_tasks:Running task rados... 2023-01-24T19:16:27.946 INFO:tasks.rados:Beginning rados... 2023-01-24T19:16:27.946 DEBUG:teuthology.run_tasks:Unwinding manager rados 2023-01-24T19:16:27.963 INFO:tasks.rados:joining rados 2023-01-24T19:16:27.964 INFO:tasks.rados:clients are ['client.0', 'client.1'] 2023-01-24T19:16:27.964 INFO:tasks.rados:starting run 0 out of 1 2023-01-24T19:16:27.964 DEBUG:teuthology.orchestra.run.smithi161:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --max-ops 4000 --objects 500 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 50 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op copy_from 50 --op cache_flush 50 --op cache_try_flush 50 --op cache_evict 50 --op write_excl 50 --pool base 2023-01-24T19:16:27.992 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight read -> 100 2023-01-24T19:16:27.993 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight write -> 50 2023-01-24T19:16:27.993 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight delete -> 50 2023-01-24T19:16:27.993 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight snap_create -> 50 2023-01-24T19:16:27.993 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight snap_remove -> 50 2023-01-24T19:16:27.994 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight rollback -> 50 2023-01-24T19:16:27.994 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight copy_from -> 50 2023-01-24T19:16:27.994 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight cache_flush -> 50 2023-01-24T19:16:27.994 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight cache_try_flush -> 50 2023-01-24T19:16:27.994 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight cache_evict -> 50 2023-01-24T19:16:27.995 INFO:tasks.rados.rados.0.smithi161.stdout:adding op weight write_excl -> 50 2023-01-24T19:16:27.995 INFO:tasks.rados.rados.0.smithi161.stdout:ceph version 17.2.5-727-g49f8fb05 (49f8fb05584886826e8eade75f7105fba754560c) quincy (stable) 2023-01-24T19:16:27.995 INFO:tasks.rados.rados.0.smithi161.stdout:Configuration: 2023-01-24T19:16:27.995 INFO:tasks.rados.rados.0.smithi161.stdout: Number of operations: 4000 2023-01-24T19:16:27.995 INFO:tasks.rados.rados.0.smithi161.stdout: Number of objects: 500 2023-01-24T19:16:27.996 INFO:tasks.rados.rados.0.smithi161.stdout: Max in flight operations: 16 2023-01-24T19:16:27.996 INFO:tasks.rados.rados.0.smithi161.stdout: Object size (in bytes): 4000000 2023-01-24T19:16:27.996 INFO:tasks.rados.rados.0.smithi161.stdout: Write stride min: 400000 2023-01-24T19:16:27.996 INFO:tasks.rados.rados.0.smithi161.stdout: Write stride max: 800000 2023-01-24T19:16:28.351 INFO:tasks.ceph.osd.5.smithi187.stderr:2023-01-24T19:16:28.350+0000 7fa7926d0700 -1 osd.5 27 handle_osd_map injecting map crc failure 2023-01-24T19:16:28.919 INFO:tasks.ceph.osd.6.smithi187.stderr:2023-01-24T19:16:28.918+0000 7f9811a09700 -1 osd.6 29 handle_osd_map injecting map crc failure 2023-01-24T19:16:28.920 INFO:tasks.rados.rados.0.smithi161.stdout:1: write initial oid 1 2023-01-24T19:16:28.920 INFO:tasks.rados.rados.0.smithi161.stdout:1: seq_num 0 ranges {416807=475249,1542129=474678} 2023-01-24T19:16:28.925 INFO:tasks.rados.rados.0.smithi161.stdout:1: writing smithi161112115-1 from 416807 to 892056 tid 1 2023-01-24T19:16:28.932 INFO:tasks.rados.rados.0.smithi161.stdout:1: writing smithi161112115-1 from 1542129 to 2016807 tid 2 2023-01-24T19:16:28.932 INFO:tasks.rados.rados.0.smithi161.stdout:2: write initial oid 2 2023-01-24T19:16:28.932 INFO:tasks.rados.rados.0.smithi161.stdout:2: seq_num 1 ranges {416807=475249,1542129=474678} 2023-01-24T19:16:28.937 INFO:tasks.rados.rados.0.smithi161.stdout:2: writing smithi161112115-2 from 416807 to 892056 tid 1 2023-01-24T19:16:28.943 INFO:tasks.rados.rados.0.smithi161.stdout:2: writing smithi161112115-2 from 1542129 to 2016807 tid 2 2023-01-24T19:16:28.944 INFO:tasks.rados.rados.0.smithi161.stdout:3: write initial oid 3 2023-01-24T19:16:28.944 INFO:tasks.rados.rados.0.smithi161.stdout:3: seq_num 2 ranges {433614=550498,1600611=433003} 2023-01-24T19:16:28.949 INFO:tasks.rados.rados.0.smithi161.stdout:3: writing smithi161112115-3 from 433614 to 984112 tid 1 2023-01-24T19:16:28.951 INFO:tasks.ceph.osd.0.smithi161.stderr:2023-01-24T19:16:28.949+0000 7fd4ef014700 -1 osd.0 29 handle_osd_map injecting map crc failure 2023-01-24T19:16:28.955 INFO:tasks.rados.rados.0.smithi161.stdout:3: writing smithi161112115-3 from 1600611 to 2033614 tid 2 2023-01-24T19:16:28.955 INFO:tasks.rados.rados.0.smithi161.stdout:4: write initial oid 4 2023-01-24T19:16:28.956 INFO:tasks.rados.rados.0.smithi161.stdout:4: seq_num 3 ranges {450421=625747,1659093=391328} 2023-01-24T19:16:28.961 INFO:tasks.rados.rados.0.smithi161.stdout:4: writing smithi161112115-4 from 450421 to 1076168 tid 1 2023-01-24T19:16:28.967 INFO:tasks.rados.rados.0.smithi161.stdout:4: writing smithi161112115-4 from 1659093 to 2050421 tid 2 2023-01-24T19:16:28.967 INFO:tasks.rados.rados.0.smithi161.stdout:5: write initial oid 5 2023-01-24T19:16:28.968 INFO:tasks.rados.rados.0.smithi161.stdout:1: finishing write tid 1 to smithi161112115-1 2023-01-24T19:16:28.968 INFO:tasks.rados.rados.0.smithi161.stdout:1: finishing write tid 2 to smithi161112115-1 2023-01-24T19:16:28.968 INFO:tasks.rados.rados.0.smithi161.stdout:5: seq_num 4 ranges {467228=700996,1717575=349653} 2023-01-24T19:16:28.974 INFO:tasks.rados.rados.0.smithi161.stdout:5: writing smithi161112115-5 from 467228 to 1168224 tid 1 2023-01-24T19:16:28.979 INFO:tasks.rados.rados.0.smithi161.stdout:5: writing smithi161112115-5 from 1717575 to 2067228 tid 2 2023-01-24T19:16:28.979 INFO:tasks.rados.rados.0.smithi161.stdout:6: write initial oid 6 2023-01-24T19:16:28.980 INFO:tasks.rados.rados.0.smithi161.stdout:6: seq_num 5 ranges {484035=776245,2059704=24331} 2023-01-24T19:16:28.986 INFO:tasks.rados.rados.0.smithi161.stdout:6: writing smithi161112115-6 from 484035 to 1260280 tid 1 2023-01-24T19:16:28.990 INFO:tasks.rados.rados.0.smithi161.stdout:6: writing smithi161112115-6 from 2059704 to 2084035 tid 2 2023-01-24T19:16:28.991 INFO:tasks.rados.rados.0.smithi161.stdout:7: write initial oid 7 2023-01-24T19:16:28.991 INFO:tasks.rados.rados.0.smithi161.stdout:7: seq_num 6 ranges {500842=451494,1718186=382656} 2023-01-24T19:16:28.996 INFO:tasks.rados.rados.0.smithi161.stdout:7: writing smithi161112115-7 from 500842 to 952336 tid 1 2023-01-24T19:16:29.002 INFO:tasks.rados.rados.0.smithi161.stdout:7: writing smithi161112115-7 from 1718186 to 2100842 tid 2 2023-01-24T19:16:29.003 INFO:tasks.rados.rados.0.smithi161.stdout:8: write initial oid 8 2023-01-24T19:16:29.003 INFO:tasks.rados.rados.0.smithi161.stdout:8: seq_num 7 ranges {517649=526743,1776668=340981} 2023-01-24T19:16:29.008 INFO:tasks.rados.rados.0.smithi161.stdout:8: writing smithi161112115-8 from 517649 to 1044392 tid 1 2023-01-24T19:16:29.015 INFO:tasks.rados.rados.0.smithi161.stdout:8: writing smithi161112115-8 from 1776668 to 2117649 tid 2 2023-01-24T19:16:29.015 INFO:tasks.rados.rados.0.smithi161.stdout:9: write initial oid 9 2023-01-24T19:16:29.015 INFO:tasks.rados.rados.0.smithi161.stdout:9: seq_num 8 ranges {534456=718345,1951503=182953} 2023-01-24T19:16:29.021 INFO:tasks.rados.rados.0.smithi161.stdout:9: writing smithi161112115-9 from 534456 to 1252801 tid 1 2023-01-24T19:16:29.027 INFO:tasks.rados.rados.0.smithi161.stdout:9: writing smithi161112115-9 from 1951503 to 2134456 tid 2 2023-01-24T19:16:29.027 INFO:tasks.rados.rados.0.smithi161.stdout:10: write initial oid 10 2023-01-24T19:16:29.027 INFO:tasks.rados.rados.0.smithi161.stdout:10: seq_num 9 ranges {551263=793594,1893632=257631} 2023-01-24T19:16:29.034 INFO:tasks.rados.rados.0.smithi161.stdout:10: writing smithi161112115-10 from 551263 to 1344857 tid 1 2023-01-24T19:16:29.039 INFO:tasks.rados.rados.0.smithi161.stdout:10: writing smithi161112115-10 from 1893632 to 2151263 tid 2 2023-01-24T19:16:29.039 INFO:tasks.rados.rados.0.smithi161.stdout:11: write initial oid 11 2023-01-24T19:16:29.039 INFO:tasks.rados.rados.0.smithi161.stdout:11: seq_num 10 ranges {568070=468843,1552114=615956} 2023-01-24T19:16:29.044 INFO:tasks.rados.rados.0.smithi161.stdout:11: writing smithi161112115-11 from 568070 to 1036913 tid 1 2023-01-24T19:16:29.050 INFO:tasks.rados.rados.0.smithi161.stdout:11: writing smithi161112115-11 from 1552114 to 2168070 tid 2 2023-01-24T19:16:29.051 INFO:tasks.rados.rados.0.smithi161.stdout:12: write initial oid 12 2023-01-24T19:16:29.051 INFO:tasks.rados.rados.0.smithi161.stdout:12: seq_num 11 ranges {584877=544092,1610596=562003} 2023-01-24T19:16:29.057 INFO:tasks.rados.rados.0.smithi161.stdout:12: writing smithi161112115-12 from 584877 to 1128969 tid 1 2023-01-24T19:16:29.063 INFO:tasks.rados.rados.0.smithi161.stdout:12: writing smithi161112115-12 from 1610596 to 2172599 tid 2 2023-01-24T19:16:29.063 INFO:tasks.rados.rados.0.smithi161.stdout:13: write initial oid 13 2023-01-24T19:16:29.063 INFO:tasks.rados.rados.0.smithi161.stdout:13: seq_num 12 ranges {601684=619341,1669078=532606} 2023-01-24T19:16:29.070 INFO:tasks.rados.rados.0.smithi161.stdout:13: writing smithi161112115-13 from 601684 to 1221025 tid 1 2023-01-24T19:16:29.076 INFO:tasks.rados.rados.0.smithi161.stdout:13: writing smithi161112115-13 from 1669078 to 2201684 tid 2 2023-01-24T19:16:29.076 INFO:tasks.rados.rados.0.smithi161.stdout:14: write initial oid 14 2023-01-24T19:16:29.076 INFO:tasks.rados.rados.0.smithi161.stdout:14: seq_num 13 ranges {618491=694590,2011207=207284} 2023-01-24T19:16:29.083 INFO:tasks.rados.rados.0.smithi161.stdout:14: writing smithi161112115-14 from 618491 to 1313081 tid 1 2023-01-24T19:16:29.088 INFO:tasks.rados.rados.0.smithi161.stdout:14: writing smithi161112115-14 from 2011207 to 2218491 tid 2 2023-01-24T19:16:29.088 INFO:tasks.rados.rados.0.smithi161.stdout:15: write initial oid 15 2023-01-24T19:16:29.089 INFO:tasks.rados.rados.0.smithi161.stdout:15: seq_num 14 ranges {635298=769839,2069689=165609} 2023-01-24T19:16:29.096 INFO:tasks.rados.rados.0.smithi161.stdout:15: writing smithi161112115-15 from 635298 to 1405137 tid 1 2023-01-24T19:16:29.101 INFO:tasks.rados.rados.0.smithi161.stdout:15: writing smithi161112115-15 from 2069689 to 2235298 tid 2 2023-01-24T19:16:29.101 INFO:tasks.rados.rados.0.smithi161.stdout:16: write initial oid 16 2023-01-24T19:16:29.101 INFO:tasks.rados.rados.0.smithi161.stdout:16: seq_num 15 ranges {652105=445088,1728171=452988} 2023-01-24T19:16:29.107 INFO:tasks.rados.rados.0.smithi161.stdout:16: writing smithi161112115-16 from 652105 to 1097193 tid 1 2023-01-24T19:16:29.114 INFO:tasks.rados.rados.0.smithi161.stdout:16: writing smithi161112115-16 from 1728171 to 2181159 tid 2 2023-01-24T19:16:29.114 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.114 INFO:tasks.rados.rados.0.smithi161.stdout:2: finishing write tid 1 to smithi161112115-2 2023-01-24T19:16:29.114 INFO:tasks.rados.rados.0.smithi161.stdout:1: finishing write tid 3 to smithi161112115-1 2023-01-24T19:16:29.114 INFO:tasks.rados.rados.0.smithi161.stdout:1: finishing write tid 4 to smithi161112115-1 2023-01-24T19:16:29.115 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 1 v 3 (ObjNum 0 snap 0 seq_num 0) dirty exists 2023-01-24T19:16:29.115 INFO:tasks.rados.rados.0.smithi161.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2023-01-24T19:16:29.115 INFO:tasks.rados.rados.0.smithi161.stdout:2: finishing write tid 2 to smithi161112115-2 2023-01-24T19:16:29.115 INFO:tasks.rados.rados.0.smithi161.stdout:2: finishing write tid 3 to smithi161112115-2 2023-01-24T19:16:29.116 INFO:tasks.rados.rados.0.smithi161.stdout:1: done (15 left) 2023-01-24T19:16:29.116 INFO:tasks.rados.rados.0.smithi161.stdout:17: write initial oid 17 2023-01-24T19:16:29.116 INFO:tasks.rados.rados.0.smithi161.stdout:17: seq_num 16 ranges {668912=636690,1903006=365906} 2023-01-24T19:16:29.121 INFO:tasks.rados.rados.0.smithi161.stdout:17: writing smithi161112115-17 from 668912 to 1305602 tid 1 2023-01-24T19:16:29.127 INFO:tasks.rados.rados.0.smithi161.stdout:17: writing smithi161112115-17 from 1903006 to 2268912 tid 2 2023-01-24T19:16:29.127 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.127 INFO:tasks.rados.rados.0.smithi161.stdout:2: finishing write tid 4 to smithi161112115-2 2023-01-24T19:16:29.128 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 2 v 3 (ObjNum 1 snap 0 seq_num 1) dirty exists 2023-01-24T19:16:29.128 INFO:tasks.rados.rados.0.smithi161.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2023-01-24T19:16:29.128 INFO:tasks.rados.rados.0.smithi161.stdout:3: finishing write tid 1 to smithi161112115-3 2023-01-24T19:16:29.128 INFO:tasks.rados.rados.0.smithi161.stdout:2: done (15 left) 2023-01-24T19:16:29.128 INFO:tasks.rados.rados.0.smithi161.stdout:18: write initial oid 18 2023-01-24T19:16:29.129 INFO:tasks.rados.rados.0.smithi161.stdout:18: seq_num 17 ranges {685719=711939,1845135=440584} 2023-01-24T19:16:29.135 INFO:tasks.rados.rados.0.smithi161.stdout:18: writing smithi161112115-18 from 685719 to 1397658 tid 1 2023-01-24T19:16:29.141 INFO:tasks.rados.rados.0.smithi161.stdout:18: writing smithi161112115-18 from 1845135 to 2285719 tid 2 2023-01-24T19:16:29.141 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.141 INFO:tasks.rados.rados.0.smithi161.stdout:3: finishing write tid 2 to smithi161112115-3 2023-01-24T19:16:29.141 INFO:tasks.rados.rados.0.smithi161.stdout:3: finishing write tid 3 to smithi161112115-3 2023-01-24T19:16:29.141 INFO:tasks.rados.rados.0.smithi161.stdout:3: finishing write tid 4 to smithi161112115-3 2023-01-24T19:16:29.142 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 3 v 3 (ObjNum 2 snap 0 seq_num 2) dirty exists 2023-01-24T19:16:29.142 INFO:tasks.rados.rados.0.smithi161.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2023-01-24T19:16:29.142 INFO:tasks.rados.rados.0.smithi161.stdout:4: finishing write tid 1 to smithi161112115-4 2023-01-24T19:16:29.142 INFO:tasks.rados.rados.0.smithi161.stdout:4: finishing write tid 2 to smithi161112115-4 2023-01-24T19:16:29.142 INFO:tasks.rados.rados.0.smithi161.stdout:4: finishing write tid 3 to smithi161112115-4 2023-01-24T19:16:29.143 INFO:tasks.rados.rados.0.smithi161.stdout:4: finishing write tid 4 to smithi161112115-4 2023-01-24T19:16:29.143 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 4 v 3 (ObjNum 3 snap 0 seq_num 3) dirty exists 2023-01-24T19:16:29.143 INFO:tasks.rados.rados.0.smithi161.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2023-01-24T19:16:29.143 INFO:tasks.rados.rados.0.smithi161.stdout:5: finishing write tid 1 to smithi161112115-5 2023-01-24T19:16:29.143 INFO:tasks.rados.rados.0.smithi161.stdout:3: done (15 left) 2023-01-24T19:16:29.144 INFO:tasks.rados.rados.0.smithi161.stdout:4: done (14 left) 2023-01-24T19:16:29.144 INFO:tasks.rados.rados.0.smithi161.stdout:19: write initial oid 19 2023-01-24T19:16:29.144 INFO:tasks.rados.rados.0.smithi161.stdout:19: seq_num 18 ranges {702526=787188,1903617=398909} 2023-01-24T19:16:29.149 INFO:tasks.rados.rados.0.smithi161.stdout:19: writing smithi161112115-19 from 702526 to 1489714 tid 1 2023-01-24T19:16:29.154 INFO:tasks.rados.rados.0.smithi161.stdout:19: writing smithi161112115-19 from 1903617 to 2302526 tid 2 2023-01-24T19:16:29.154 INFO:tasks.rados.rados.0.smithi161.stdout:20: write initial oid 20 2023-01-24T19:16:29.154 INFO:tasks.rados.rados.0.smithi161.stdout:20: seq_num 19 ranges {719333=462437,1962099=357234} 2023-01-24T19:16:29.160 INFO:tasks.rados.rados.0.smithi161.stdout:20: writing smithi161112115-20 from 719333 to 1181770 tid 1 2023-01-24T19:16:29.167 INFO:tasks.rados.rados.0.smithi161.stdout:20: writing smithi161112115-20 from 1962099 to 2319333 tid 2 2023-01-24T19:16:29.167 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.168 INFO:tasks.rados.rados.0.smithi161.stdout:6: finishing write tid 1 to smithi161112115-6 2023-01-24T19:16:29.168 INFO:tasks.rados.rados.0.smithi161.stdout:5: finishing write tid 2 to smithi161112115-5 2023-01-24T19:16:29.168 INFO:tasks.rados.rados.0.smithi161.stdout:5: finishing write tid 3 to smithi161112115-5 2023-01-24T19:16:29.168 INFO:tasks.rados.rados.0.smithi161.stdout:5: finishing write tid 4 to smithi161112115-5 2023-01-24T19:16:29.169 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 5 v 6 (ObjNum 4 snap 0 seq_num 4) dirty exists 2023-01-24T19:16:29.169 INFO:tasks.rados.rados.0.smithi161.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2023-01-24T19:16:29.169 INFO:tasks.rados.rados.0.smithi161.stdout:6: finishing write tid 2 to smithi161112115-6 2023-01-24T19:16:29.169 INFO:tasks.rados.rados.0.smithi161.stdout:6: finishing write tid 3 to smithi161112115-6 2023-01-24T19:16:29.169 INFO:tasks.rados.rados.0.smithi161.stdout:5: done (15 left) 2023-01-24T19:16:29.170 INFO:tasks.rados.rados.0.smithi161.stdout:21: write initial oid 21 2023-01-24T19:16:29.170 INFO:tasks.rados.rados.0.smithi161.stdout:21: seq_num 20 ranges {736140=537686,2020581=315559} 2023-01-24T19:16:29.174 INFO:tasks.rados.rados.0.smithi161.stdout:21: writing smithi161112115-21 from 736140 to 1273826 tid 1 2023-01-24T19:16:29.180 INFO:tasks.rados.rados.0.smithi161.stdout:21: writing smithi161112115-21 from 2020581 to 2336140 tid 2 2023-01-24T19:16:29.180 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.181 INFO:tasks.rados.rados.0.smithi161.stdout:7: finishing write tid 1 to smithi161112115-7 2023-01-24T19:16:29.181 INFO:tasks.rados.rados.0.smithi161.stdout:6: finishing write tid 4 to smithi161112115-6 2023-01-24T19:16:29.181 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 6 v 6 (ObjNum 5 snap 0 seq_num 5) dirty exists 2023-01-24T19:16:29.181 INFO:tasks.rados.rados.0.smithi161.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2023-01-24T19:16:29.182 INFO:tasks.rados.rados.0.smithi161.stdout:7: finishing write tid 2 to smithi161112115-7 2023-01-24T19:16:29.182 INFO:tasks.rados.rados.0.smithi161.stdout:6: done (15 left) 2023-01-24T19:16:29.182 INFO:tasks.rados.rados.0.smithi161.stdout:22: write initial oid 22 2023-01-24T19:16:29.182 INFO:tasks.rados.rados.0.smithi161.stdout:22: seq_num 21 ranges {752947=612935,1962710=390237} 2023-01-24T19:16:29.187 INFO:tasks.rados.rados.0.smithi161.stdout:22: writing smithi161112115-22 from 752947 to 1365882 tid 1 2023-01-24T19:16:29.193 INFO:tasks.rados.rados.0.smithi161.stdout:22: writing smithi161112115-22 from 1962710 to 2352947 tid 2 2023-01-24T19:16:29.194 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.194 INFO:tasks.rados.rados.0.smithi161.stdout:7: finishing write tid 3 to smithi161112115-7 2023-01-24T19:16:29.194 INFO:tasks.rados.rados.0.smithi161.stdout:7: finishing write tid 4 to smithi161112115-7 2023-01-24T19:16:29.194 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 7 v 6 (ObjNum 6 snap 0 seq_num 6) dirty exists 2023-01-24T19:16:29.194 INFO:tasks.rados.rados.0.smithi161.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2023-01-24T19:16:29.195 INFO:tasks.rados.rados.0.smithi161.stdout:8: finishing write tid 1 to smithi161112115-8 2023-01-24T19:16:29.195 INFO:tasks.rados.rados.0.smithi161.stdout:8: finishing write tid 2 to smithi161112115-8 2023-01-24T19:16:29.195 INFO:tasks.rados.rados.0.smithi161.stdout:7: done (15 left) 2023-01-24T19:16:29.195 INFO:tasks.rados.rados.0.smithi161.stdout:23: write initial oid 23 2023-01-24T19:16:29.195 INFO:tasks.rados.rados.0.smithi161.stdout:23: seq_num 22 ranges {769754=688184,2021192=348562} 2023-01-24T19:16:29.202 INFO:tasks.rados.rados.0.smithi161.stdout:23: writing smithi161112115-23 from 769754 to 1457938 tid 1 2023-01-24T19:16:29.207 INFO:tasks.rados.rados.0.smithi161.stdout:23: writing smithi161112115-23 from 2021192 to 2369754 tid 2 2023-01-24T19:16:29.207 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.207 INFO:tasks.rados.rados.0.smithi161.stdout:8: finishing write tid 3 to smithi161112115-8 2023-01-24T19:16:29.208 INFO:tasks.rados.rados.0.smithi161.stdout:8: finishing write tid 4 to smithi161112115-8 2023-01-24T19:16:29.208 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 8 v 9 (ObjNum 7 snap 0 seq_num 7) dirty exists 2023-01-24T19:16:29.208 INFO:tasks.rados.rados.0.smithi161.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2023-01-24T19:16:29.208 INFO:tasks.rados.rados.0.smithi161.stdout:9: finishing write tid 1 to smithi161112115-9 2023-01-24T19:16:29.208 INFO:tasks.rados.rados.0.smithi161.stdout:9: finishing write tid 2 to smithi161112115-9 2023-01-24T19:16:29.209 INFO:tasks.rados.rados.0.smithi161.stdout:9: finishing write tid 3 to smithi161112115-9 2023-01-24T19:16:29.209 INFO:tasks.rados.rados.0.smithi161.stdout:9: finishing write tid 4 to smithi161112115-9 2023-01-24T19:16:29.209 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 9 v 12 (ObjNum 8 snap 0 seq_num 8) dirty exists 2023-01-24T19:16:29.209 INFO:tasks.rados.rados.0.smithi161.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2023-01-24T19:16:29.209 INFO:tasks.rados.rados.0.smithi161.stdout:10: finishing write tid 1 to smithi161112115-10 2023-01-24T19:16:29.210 INFO:tasks.rados.rados.0.smithi161.stdout:10: finishing write tid 2 to smithi161112115-10 2023-01-24T19:16:29.210 INFO:tasks.rados.rados.0.smithi161.stdout:10: finishing write tid 3 to smithi161112115-10 2023-01-24T19:16:29.210 INFO:tasks.rados.rados.0.smithi161.stdout:10: finishing write tid 4 to smithi161112115-10 2023-01-24T19:16:29.210 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 10 v 9 (ObjNum 9 snap 0 seq_num 9) dirty exists 2023-01-24T19:16:29.210 INFO:tasks.rados.rados.0.smithi161.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2023-01-24T19:16:29.211 INFO:tasks.rados.rados.0.smithi161.stdout:11: finishing write tid 1 to smithi161112115-11 2023-01-24T19:16:29.211 INFO:tasks.rados.rados.0.smithi161.stdout:8: done (15 left) 2023-01-24T19:16:29.211 INFO:tasks.rados.rados.0.smithi161.stdout:9: done (14 left) 2023-01-24T19:16:29.211 INFO:tasks.rados.rados.0.smithi161.stdout:10: done (13 left) 2023-01-24T19:16:29.211 INFO:tasks.rados.rados.0.smithi161.stdout:24: write initial oid 24 2023-01-24T19:16:29.212 INFO:tasks.rados.rados.0.smithi161.stdout:24: seq_num 23 ranges {786561=479786,1796027=467664} 2023-01-24T19:16:29.213 INFO:tasks.rados.rados.0.smithi161.stdout:24: writing smithi161112115-24 from 786561 to 1266347 tid 1 2023-01-24T19:16:29.219 INFO:tasks.rados.rados.0.smithi161.stdout:24: writing smithi161112115-24 from 1796027 to 2263691 tid 2 2023-01-24T19:16:29.220 INFO:tasks.rados.rados.0.smithi161.stdout:25: write initial oid 25 2023-01-24T19:16:29.220 INFO:tasks.rados.rados.0.smithi161.stdout:25: seq_num 24 ranges {403368=555035,1454509=727675} 2023-01-24T19:16:29.224 INFO:tasks.rados.rados.0.smithi161.stdout:25: writing smithi161112115-25 from 403368 to 958403 tid 1 2023-01-24T19:16:29.231 INFO:tasks.rados.rados.0.smithi161.stdout:25: writing smithi161112115-25 from 1454509 to 2182184 tid 2 2023-01-24T19:16:29.232 INFO:tasks.rados.rados.0.smithi161.stdout:26: write initial oid 26 2023-01-24T19:16:29.232 INFO:tasks.rados.rados.0.smithi161.stdout:26: seq_num 25 ranges {420175=630284,1796638=471333} 2023-01-24T19:16:29.239 INFO:tasks.rados.rados.0.smithi161.stdout:26: writing smithi161112115-26 from 420175 to 1050459 tid 1 2023-01-24T19:16:29.245 INFO:tasks.rados.rados.0.smithi161.stdout:26: writing smithi161112115-26 from 1796638 to 2267971 tid 2 2023-01-24T19:16:29.245 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.246 INFO:tasks.rados.rados.0.smithi161.stdout:12: finishing write tid 1 to smithi161112115-12 2023-01-24T19:16:29.246 INFO:tasks.rados.rados.0.smithi161.stdout:11: finishing write tid 2 to smithi161112115-11 2023-01-24T19:16:29.246 INFO:tasks.rados.rados.0.smithi161.stdout:11: finishing write tid 3 to smithi161112115-11 2023-01-24T19:16:29.246 INFO:tasks.rados.rados.0.smithi161.stdout:11: finishing write tid 4 to smithi161112115-11 2023-01-24T19:16:29.246 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 11 v 12 (ObjNum 10 snap 0 seq_num 10) dirty exists 2023-01-24T19:16:29.247 INFO:tasks.rados.rados.0.smithi161.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2023-01-24T19:16:29.247 INFO:tasks.rados.rados.0.smithi161.stdout:12: finishing write tid 2 to smithi161112115-12 2023-01-24T19:16:29.247 INFO:tasks.rados.rados.0.smithi161.stdout:12: finishing write tid 3 to smithi161112115-12 2023-01-24T19:16:29.247 INFO:tasks.rados.rados.0.smithi161.stdout:12: finishing write tid 4 to smithi161112115-12 2023-01-24T19:16:29.247 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 12 v 6 (ObjNum 11 snap 0 seq_num 11) dirty exists 2023-01-24T19:16:29.248 INFO:tasks.rados.rados.0.smithi161.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2023-01-24T19:16:29.248 INFO:tasks.rados.rados.0.smithi161.stdout:13: finishing write tid 1 to smithi161112115-13 2023-01-24T19:16:29.248 INFO:tasks.rados.rados.0.smithi161.stdout:13: finishing write tid 2 to smithi161112115-13 2023-01-24T19:16:29.248 INFO:tasks.rados.rados.0.smithi161.stdout:13: finishing write tid 3 to smithi161112115-13 2023-01-24T19:16:29.248 INFO:tasks.rados.rados.0.smithi161.stdout:13: finishing write tid 4 to smithi161112115-13 2023-01-24T19:16:29.249 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 13 v 9 (ObjNum 12 snap 0 seq_num 12) dirty exists 2023-01-24T19:16:29.249 INFO:tasks.rados.rados.0.smithi161.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2023-01-24T19:16:29.249 INFO:tasks.rados.rados.0.smithi161.stdout:14: finishing write tid 1 to smithi161112115-14 2023-01-24T19:16:29.249 INFO:tasks.rados.rados.0.smithi161.stdout:14: finishing write tid 2 to smithi161112115-14 2023-01-24T19:16:29.249 INFO:tasks.rados.rados.0.smithi161.stdout:14: finishing write tid 3 to smithi161112115-14 2023-01-24T19:16:29.250 INFO:tasks.rados.rados.0.smithi161.stdout:11: done (15 left) 2023-01-24T19:16:29.250 INFO:tasks.rados.rados.0.smithi161.stdout:12: done (14 left) 2023-01-24T19:16:29.250 INFO:tasks.rados.rados.0.smithi161.stdout:13: done (13 left) 2023-01-24T19:16:29.250 INFO:tasks.rados.rados.0.smithi161.stdout:27: write initial oid 27 2023-01-24T19:16:29.250 INFO:tasks.rados.rados.0.smithi161.stdout:27: seq_num 26 ranges {436982=705533,1855120=581862} 2023-01-24T19:16:29.251 INFO:tasks.rados.rados.0.smithi161.stdout:27: writing smithi161112115-27 from 436982 to 1142515 tid 1 2023-01-24T19:16:29.259 INFO:tasks.rados.rados.0.smithi161.stdout:27: writing smithi161112115-27 from 1855120 to 2436982 tid 2 2023-01-24T19:16:29.259 INFO:tasks.rados.rados.0.smithi161.stdout:28: write initial oid 28 2023-01-24T19:16:29.259 INFO:tasks.rados.rados.0.smithi161.stdout:14: finishing write tid 4 to smithi161112115-14 2023-01-24T19:16:29.259 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 14 v 12 (ObjNum 13 snap 0 seq_num 13) dirty exists 2023-01-24T19:16:29.260 INFO:tasks.rados.rados.0.smithi161.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2023-01-24T19:16:29.260 INFO:tasks.rados.rados.0.smithi161.stdout:15: finishing write tid 1 to smithi161112115-15 2023-01-24T19:16:29.260 INFO:tasks.rados.rados.0.smithi161.stdout:15: finishing write tid 2 to smithi161112115-15 2023-01-24T19:16:29.260 INFO:tasks.rados.rados.0.smithi161.stdout:15: finishing write tid 3 to smithi161112115-15 2023-01-24T19:16:29.260 INFO:tasks.rados.rados.0.smithi161.stdout:28: seq_num 27 ranges {453789=780782,1913602=475002} 2023-01-24T19:16:29.265 INFO:tasks.rados.rados.0.smithi161.stdout:28: writing smithi161112115-28 from 453789 to 1234571 tid 1 2023-01-24T19:16:29.272 INFO:tasks.rados.rados.0.smithi161.stdout:28: writing smithi161112115-28 from 1913602 to 2388604 tid 2 2023-01-24T19:16:29.272 INFO:tasks.rados.rados.0.smithi161.stdout:14: done (14 left) 2023-01-24T19:16:29.272 INFO:tasks.rados.rados.0.smithi161.stdout:29: write initial oid 29 2023-01-24T19:16:29.273 INFO:tasks.rados.rados.0.smithi161.stdout:29: seq_num 28 ranges {470596=456031,1572084=618660} 2023-01-24T19:16:29.277 INFO:tasks.rados.rados.0.smithi161.stdout:29: writing smithi161112115-29 from 470596 to 926627 tid 1 2023-01-24T19:16:29.284 INFO:tasks.rados.rados.0.smithi161.stdout:29: writing smithi161112115-29 from 1572084 to 2190744 tid 2 2023-01-24T19:16:29.284 INFO:tasks.rados.rados.0.smithi161.stdout:30: write initial oid 30 2023-01-24T19:16:29.284 INFO:tasks.rados.rados.0.smithi161.stdout:30: seq_num 29 ranges {487403=531280,1514213=478671} 2023-01-24T19:16:29.289 INFO:tasks.rados.rados.0.smithi161.stdout:30: writing smithi161112115-30 from 487403 to 1018683 tid 1 2023-01-24T19:16:29.295 INFO:tasks.rados.rados.0.smithi161.stdout:30: writing smithi161112115-30 from 1514213 to 1992884 tid 2 2023-01-24T19:16:29.296 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.296 INFO:tasks.rados.rados.0.smithi161.stdout:15: finishing write tid 4 to smithi161112115-15 2023-01-24T19:16:29.296 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 15 v 15 (ObjNum 14 snap 0 seq_num 14) dirty exists 2023-01-24T19:16:29.296 INFO:tasks.rados.rados.0.smithi161.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2023-01-24T19:16:29.297 INFO:tasks.rados.rados.0.smithi161.stdout:16: finishing write tid 1 to smithi161112115-16 2023-01-24T19:16:29.297 INFO:tasks.rados.rados.0.smithi161.stdout:16: finishing write tid 2 to smithi161112115-16 2023-01-24T19:16:29.297 INFO:tasks.rados.rados.0.smithi161.stdout:15: done (15 left) 2023-01-24T19:16:29.297 INFO:tasks.rados.rados.0.smithi161.stdout:31: write initial oid 31 2023-01-24T19:16:29.297 INFO:tasks.rados.rados.0.smithi161.stdout:16: finishing write tid 3 to smithi161112115-16 2023-01-24T19:16:29.298 INFO:tasks.rados.rados.0.smithi161.stdout:16: finishing write tid 4 to smithi161112115-16 2023-01-24T19:16:29.298 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 16 v 9 (ObjNum 15 snap 0 seq_num 15) dirty exists 2023-01-24T19:16:29.298 INFO:tasks.rados.rados.0.smithi161.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2023-01-24T19:16:29.298 INFO:tasks.rados.rados.0.smithi161.stdout:17: finishing write tid 1 to smithi161112115-17 2023-01-24T19:16:29.298 INFO:tasks.rados.rados.0.smithi161.stdout:17: finishing write tid 2 to smithi161112115-17 2023-01-24T19:16:29.299 INFO:tasks.rados.rados.0.smithi161.stdout:17: finishing write tid 3 to smithi161112115-17 2023-01-24T19:16:29.299 INFO:tasks.rados.rados.0.smithi161.stdout:31: seq_num 30 ranges {504210=606529,1572695=622329} 2023-01-24T19:16:29.301 INFO:tasks.rados.rados.0.smithi161.stdout:31: writing smithi161112115-31 from 504210 to 1110739 tid 1 2023-01-24T19:16:29.307 INFO:tasks.rados.rados.0.smithi161.stdout:31: writing smithi161112115-31 from 1572695 to 2195024 tid 2 2023-01-24T19:16:29.308 INFO:tasks.rados.rados.0.smithi161.stdout:16: done (15 left) 2023-01-24T19:16:29.308 INFO:tasks.rados.rados.0.smithi161.stdout:32: write initial oid 32 2023-01-24T19:16:29.308 INFO:tasks.rados.rados.0.smithi161.stdout:17: finishing write tid 4 to smithi161112115-17 2023-01-24T19:16:29.308 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 17 v 15 (ObjNum 16 snap 0 seq_num 16) dirty exists 2023-01-24T19:16:29.309 INFO:tasks.rados.rados.0.smithi161.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2023-01-24T19:16:29.309 INFO:tasks.rados.rados.0.smithi161.stdout:32: seq_num 31 ranges {521017=798131,1747530=482340} 2023-01-24T19:16:29.314 INFO:tasks.rados.rados.0.smithi161.stdout:32: writing smithi161112115-32 from 521017 to 1319148 tid 1 2023-01-24T19:16:29.320 INFO:tasks.rados.rados.0.smithi161.stdout:32: writing smithi161112115-32 from 1747530 to 2229870 tid 2 2023-01-24T19:16:29.320 INFO:tasks.rados.rados.0.smithi161.stdout:17: done (15 left) 2023-01-24T19:16:29.321 INFO:tasks.rados.rados.0.smithi161.stdout:33: write initial oid 33 2023-01-24T19:16:29.321 INFO:tasks.rados.rados.0.smithi161.stdout:33: seq_num 32 ranges {537824=473380,1806012=625998} 2023-01-24T19:16:29.325 INFO:tasks.rados.rados.0.smithi161.stdout:33: writing smithi161112115-33 from 537824 to 1011204 tid 1 2023-01-24T19:16:29.334 INFO:tasks.rados.rados.0.smithi161.stdout:33: writing smithi161112115-33 from 1806012 to 2432010 tid 2 2023-01-24T19:16:29.334 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.334 INFO:tasks.rados.rados.0.smithi161.stdout:18: finishing write tid 1 to smithi161112115-18 2023-01-24T19:16:29.335 INFO:tasks.rados.rados.0.smithi161.stdout:18: finishing write tid 2 to smithi161112115-18 2023-01-24T19:16:29.335 INFO:tasks.rados.rados.0.smithi161.stdout:18: finishing write tid 3 to smithi161112115-18 2023-01-24T19:16:29.335 INFO:tasks.rados.rados.0.smithi161.stdout:18: finishing write tid 4 to smithi161112115-18 2023-01-24T19:16:29.335 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 18 v 18 (ObjNum 17 snap 0 seq_num 17) dirty exists 2023-01-24T19:16:29.335 INFO:tasks.rados.rados.0.smithi161.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2023-01-24T19:16:29.336 INFO:tasks.rados.rados.0.smithi161.stdout:19: finishing write tid 1 to smithi161112115-19 2023-01-24T19:16:29.336 INFO:tasks.rados.rados.0.smithi161.stdout:19: finishing write tid 2 to smithi161112115-19 2023-01-24T19:16:29.336 INFO:tasks.rados.rados.0.smithi161.stdout:18: done (15 left) 2023-01-24T19:16:29.336 INFO:tasks.rados.rados.0.smithi161.stdout:34: write initial oid 34 2023-01-24T19:16:29.337 INFO:tasks.rados.rados.0.smithi161.stdout:34: seq_num 33 ranges {554631=548629,1748141=486009} 2023-01-24T19:16:29.339 INFO:tasks.rados.rados.0.smithi161.stdout:34: writing smithi161112115-34 from 554631 to 1103260 tid 1 2023-01-24T19:16:29.346 INFO:tasks.rados.rados.0.smithi161.stdout:34: writing smithi161112115-34 from 1748141 to 2234150 tid 2 2023-01-24T19:16:29.346 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.347 INFO:tasks.rados.rados.0.smithi161.stdout:19: finishing write tid 3 to smithi161112115-19 2023-01-24T19:16:29.347 INFO:tasks.rados.rados.0.smithi161.stdout:19: finishing write tid 4 to smithi161112115-19 2023-01-24T19:16:29.347 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 19 v 18 (ObjNum 18 snap 0 seq_num 18) dirty exists 2023-01-24T19:16:29.347 INFO:tasks.rados.rados.0.smithi161.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2023-01-24T19:16:29.347 INFO:tasks.rados.rados.0.smithi161.stdout:20: finishing write tid 1 to smithi161112115-20 2023-01-24T19:16:29.348 INFO:tasks.rados.rados.0.smithi161.stdout:20: finishing write tid 2 to smithi161112115-20 2023-01-24T19:16:29.348 INFO:tasks.rados.rados.0.smithi161.stdout:20: finishing write tid 3 to smithi161112115-20 2023-01-24T19:16:29.348 INFO:tasks.rados.rados.0.smithi161.stdout:20: finishing write tid 4 to smithi161112115-20 2023-01-24T19:16:29.348 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 20 v 12 (ObjNum 19 snap 0 seq_num 19) dirty exists 2023-01-24T19:16:29.348 INFO:tasks.rados.rados.0.smithi161.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2023-01-24T19:16:29.349 INFO:tasks.rados.rados.0.smithi161.stdout:21: finishing write tid 1 to smithi161112115-21 2023-01-24T19:16:29.349 INFO:tasks.rados.rados.0.smithi161.stdout:19: done (15 left) 2023-01-24T19:16:29.349 INFO:tasks.rados.rados.0.smithi161.stdout:20: done (14 left) 2023-01-24T19:16:29.349 INFO:tasks.rados.rados.0.smithi161.stdout:35: write initial oid 35 2023-01-24T19:16:29.349 INFO:tasks.rados.rados.0.smithi161.stdout:35: seq_num 34 ranges {571438=623878,1806623=629667} 2023-01-24T19:16:29.352 INFO:tasks.rados.rados.0.smithi161.stdout:35: writing smithi161112115-35 from 571438 to 1195316 tid 1 2023-01-24T19:16:29.360 INFO:tasks.rados.rados.0.smithi161.stdout:35: writing smithi161112115-35 from 1806623 to 2436290 tid 2 2023-01-24T19:16:29.361 INFO:tasks.rados.rados.0.smithi161.stdout:36: write initial oid 36 2023-01-24T19:16:29.361 INFO:tasks.rados.rados.0.smithi161.stdout:21: finishing write tid 2 to smithi161112115-21 2023-01-24T19:16:29.362 INFO:tasks.rados.rados.0.smithi161.stdout:21: finishing write tid 3 to smithi161112115-21 2023-01-24T19:16:29.362 INFO:tasks.rados.rados.0.smithi161.stdout:21: finishing write tid 4 to smithi161112115-21 2023-01-24T19:16:29.362 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 21 v 21 (ObjNum 20 snap 0 seq_num 20) dirty exists 2023-01-24T19:16:29.362 INFO:tasks.rados.rados.0.smithi161.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2023-01-24T19:16:29.362 INFO:tasks.rados.rados.0.smithi161.stdout:22: finishing write tid 1 to smithi161112115-22 2023-01-24T19:16:29.363 INFO:tasks.rados.rados.0.smithi161.stdout:36: seq_num 35 ranges {588245=699127,1865105=489678} 2023-01-24T19:16:29.367 INFO:tasks.rados.rados.0.smithi161.stdout:36: writing smithi161112115-36 from 588245 to 1287372 tid 1 2023-01-24T19:16:29.373 INFO:tasks.rados.rados.0.smithi161.stdout:36: writing smithi161112115-36 from 1865105 to 2354783 tid 2 2023-01-24T19:16:29.373 INFO:tasks.rados.rados.0.smithi161.stdout:21: done (15 left) 2023-01-24T19:16:29.374 INFO:tasks.rados.rados.0.smithi161.stdout:37: write initial oid 37 2023-01-24T19:16:29.374 INFO:tasks.rados.rados.0.smithi161.stdout:37: seq_num 36 ranges {605052=774376,1923587=633336} 2023-01-24T19:16:29.381 INFO:tasks.rados.rados.0.smithi161.stdout:37: writing smithi161112115-37 from 605052 to 1379428 tid 1 2023-01-24T19:16:29.388 INFO:tasks.rados.rados.0.smithi161.stdout:37: writing smithi161112115-37 from 1923587 to 2556923 tid 2 2023-01-24T19:16:29.388 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.389 INFO:tasks.rados.rados.0.smithi161.stdout:22: finishing write tid 2 to smithi161112115-22 2023-01-24T19:16:29.389 INFO:tasks.rados.rados.0.smithi161.stdout:22: finishing write tid 3 to smithi161112115-22 2023-01-24T19:16:29.389 INFO:tasks.rados.rados.0.smithi161.stdout:22: finishing write tid 4 to smithi161112115-22 2023-01-24T19:16:29.389 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 22 v 24 (ObjNum 21 snap 0 seq_num 21) dirty exists 2023-01-24T19:16:29.389 INFO:tasks.rados.rados.0.smithi161.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2023-01-24T19:16:29.389 INFO:tasks.rados.rados.0.smithi161.stdout:23: finishing write tid 1 to smithi161112115-23 2023-01-24T19:16:29.390 INFO:tasks.rados.rados.0.smithi161.stdout:22: done (15 left) 2023-01-24T19:16:29.390 INFO:tasks.rados.rados.0.smithi161.stdout:38: write initial oid 38 2023-01-24T19:16:29.390 INFO:tasks.rados.rados.0.smithi161.stdout:38: seq_num 37 ranges {621859=449625,1865716=756143} 2023-01-24T19:16:29.394 INFO:tasks.rados.rados.0.smithi161.stdout:38: writing smithi161112115-38 from 621859 to 1071484 tid 1 2023-01-24T19:16:29.403 INFO:tasks.rados.rados.0.smithi161.stdout:38: writing smithi161112115-38 from 1865716 to 2621859 tid 2 2023-01-24T19:16:29.404 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.404 INFO:tasks.rados.rados.0.smithi161.stdout:23: finishing write tid 2 to smithi161112115-23 2023-01-24T19:16:29.404 INFO:tasks.rados.rados.0.smithi161.stdout:23: finishing write tid 3 to smithi161112115-23 2023-01-24T19:16:29.404 INFO:tasks.rados.rados.0.smithi161.stdout:23: finishing write tid 4 to smithi161112115-23 2023-01-24T19:16:29.404 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 23 v 15 (ObjNum 22 snap 0 seq_num 22) dirty exists 2023-01-24T19:16:29.405 INFO:tasks.rados.rados.0.smithi161.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2023-01-24T19:16:29.405 INFO:tasks.rados.rados.0.smithi161.stdout:24: finishing write tid 1 to smithi161112115-24 2023-01-24T19:16:29.405 INFO:tasks.rados.rados.0.smithi161.stdout:23: done (15 left) 2023-01-24T19:16:29.405 INFO:tasks.rados.rados.0.smithi161.stdout:39: write initial oid 39 2023-01-24T19:16:29.405 INFO:tasks.rados.rados.0.smithi161.stdout:39: seq_num 38 ranges {638666=524874,1924198=637005} 2023-01-24T19:16:29.409 INFO:tasks.rados.rados.0.smithi161.stdout:39: writing smithi161112115-39 from 638666 to 1163540 tid 1 2023-01-24T19:16:29.418 INFO:tasks.rados.rados.0.smithi161.stdout:39: writing smithi161112115-39 from 1924198 to 2561203 tid 2 2023-01-24T19:16:29.418 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.418 INFO:tasks.rados.rados.0.smithi161.stdout:24: finishing write tid 2 to smithi161112115-24 2023-01-24T19:16:29.418 INFO:tasks.rados.rados.0.smithi161.stdout:24: finishing write tid 3 to smithi161112115-24 2023-01-24T19:16:29.419 INFO:tasks.rados.rados.0.smithi161.stdout:24: finishing write tid 4 to smithi161112115-24 2023-01-24T19:16:29.419 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 24 v 15 (ObjNum 23 snap 0 seq_num 23) dirty exists 2023-01-24T19:16:29.419 INFO:tasks.rados.rados.0.smithi161.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2023-01-24T19:16:29.419 INFO:tasks.rados.rados.0.smithi161.stdout:25: finishing write tid 1 to smithi161112115-25 2023-01-24T19:16:29.419 INFO:tasks.rados.rados.0.smithi161.stdout:25: finishing write tid 2 to smithi161112115-25 2023-01-24T19:16:29.420 INFO:tasks.rados.rados.0.smithi161.stdout:24: done (15 left) 2023-01-24T19:16:29.420 INFO:tasks.rados.rados.0.smithi161.stdout:40: write initial oid 40 2023-01-24T19:16:29.420 INFO:tasks.rados.rados.0.smithi161.stdout:40: seq_num 39 ranges {655473=716476,2099033=556440} 2023-01-24T19:16:29.425 INFO:tasks.rados.rados.0.smithi161.stdout:40: writing smithi161112115-40 from 655473 to 1371949 tid 1 2023-01-24T19:16:29.433 INFO:tasks.rados.rados.0.smithi161.stdout:40: writing smithi161112115-40 from 2099033 to 2655473 tid 2 2023-01-24T19:16:29.433 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.433 INFO:tasks.rados.rados.0.smithi161.stdout:25: finishing write tid 3 to smithi161112115-25 2023-01-24T19:16:29.433 INFO:tasks.rados.rados.0.smithi161.stdout:25: finishing write tid 4 to smithi161112115-25 2023-01-24T19:16:29.434 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 25 v 18 (ObjNum 24 snap 0 seq_num 24) dirty exists 2023-01-24T19:16:29.434 INFO:tasks.rados.rados.0.smithi161.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2023-01-24T19:16:29.434 INFO:tasks.rados.rados.0.smithi161.stdout:26: finishing write tid 1 to smithi161112115-26 2023-01-24T19:16:29.434 INFO:tasks.rados.rados.0.smithi161.stdout:26: finishing write tid 2 to smithi161112115-26 2023-01-24T19:16:29.434 INFO:tasks.rados.rados.0.smithi161.stdout:25: done (15 left) 2023-01-24T19:16:29.435 INFO:tasks.rados.rados.0.smithi161.stdout:41: write initial oid 41 2023-01-24T19:16:29.435 INFO:tasks.rados.rados.0.smithi161.stdout:41: seq_num 40 ranges {672280=791725,2157515=514765} 2023-01-24T19:16:29.441 INFO:tasks.rados.rados.0.smithi161.stdout:41: writing smithi161112115-41 from 672280 to 1464005 tid 1 2023-01-24T19:16:29.448 INFO:tasks.rados.rados.0.smithi161.stdout:41: writing smithi161112115-41 from 2157515 to 2672280 tid 2 2023-01-24T19:16:29.448 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.448 INFO:tasks.rados.rados.0.smithi161.stdout:26: finishing write tid 3 to smithi161112115-26 2023-01-24T19:16:29.449 INFO:tasks.rados.rados.0.smithi161.stdout:26: finishing write tid 4 to smithi161112115-26 2023-01-24T19:16:29.449 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 26 v 27 (ObjNum 25 snap 0 seq_num 25) dirty exists 2023-01-24T19:16:29.449 INFO:tasks.rados.rados.0.smithi161.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2023-01-24T19:16:29.449 INFO:tasks.rados.rados.0.smithi161.stdout:27: finishing write tid 1 to smithi161112115-27 2023-01-24T19:16:29.449 INFO:tasks.rados.rados.0.smithi161.stdout:26: done (15 left) 2023-01-24T19:16:29.450 INFO:tasks.rados.rados.0.smithi161.stdout:42: write initial oid 42 2023-01-24T19:16:29.450 INFO:tasks.rados.rados.0.smithi161.stdout:42: seq_num 41 ranges {689087=466974,1699644=784332} 2023-01-24T19:16:29.454 INFO:tasks.rados.rados.0.smithi161.stdout:42: writing smithi161112115-42 from 689087 to 1156061 tid 1 2023-01-24T19:16:29.462 INFO:tasks.rados.rados.0.smithi161.stdout:42: writing smithi161112115-42 from 1699644 to 2483976 tid 2 2023-01-24T19:16:29.462 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.462 INFO:tasks.rados.rados.0.smithi161.stdout:27: finishing write tid 2 to smithi161112115-27 2023-01-24T19:16:29.463 INFO:tasks.rados.rados.0.smithi161.stdout:27: finishing write tid 3 to smithi161112115-27 2023-01-24T19:16:29.463 INFO:tasks.rados.rados.0.smithi161.stdout:27: finishing write tid 4 to smithi161112115-27 2023-01-24T19:16:29.463 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 27 v 12 (ObjNum 26 snap 0 seq_num 26) dirty exists 2023-01-24T19:16:29.463 INFO:tasks.rados.rados.0.smithi161.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2023-01-24T19:16:29.463 INFO:tasks.rados.rados.0.smithi161.stdout:28: finishing write tid 1 to smithi161112115-28 2023-01-24T19:16:29.464 INFO:tasks.rados.rados.0.smithi161.stdout:29: finishing write tid 1 to smithi161112115-29 2023-01-24T19:16:29.464 INFO:tasks.rados.rados.0.smithi161.stdout:28: finishing write tid 2 to smithi161112115-28 2023-01-24T19:16:29.464 INFO:tasks.rados.rados.0.smithi161.stdout:28: finishing write tid 3 to smithi161112115-28 2023-01-24T19:16:29.464 INFO:tasks.rados.rados.0.smithi161.stdout:28: finishing write tid 4 to smithi161112115-28 2023-01-24T19:16:29.464 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 28 v 21 (ObjNum 27 snap 0 seq_num 27) dirty exists 2023-01-24T19:16:29.465 INFO:tasks.rados.rados.0.smithi161.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2023-01-24T19:16:29.465 INFO:tasks.rados.rados.0.smithi161.stdout:29: finishing write tid 2 to smithi161112115-29 2023-01-24T19:16:29.465 INFO:tasks.rados.rados.0.smithi161.stdout:27: done (15 left) 2023-01-24T19:16:29.465 INFO:tasks.rados.rados.0.smithi161.stdout:28: done (14 left) 2023-01-24T19:16:29.466 INFO:tasks.rados.rados.0.smithi161.stdout:43: write initial oid 43 2023-01-24T19:16:29.466 INFO:tasks.rados.rados.0.smithi161.stdout:43: seq_num 42 ranges {705894=542223,1758126=644343} 2023-01-24T19:16:29.468 INFO:tasks.rados.rados.0.smithi161.stdout:43: writing smithi161112115-43 from 705894 to 1248117 tid 1 2023-01-24T19:16:29.476 INFO:tasks.rados.rados.0.smithi161.stdout:43: writing smithi161112115-43 from 1758126 to 2402469 tid 2 2023-01-24T19:16:29.476 INFO:tasks.rados.rados.0.smithi161.stdout:44: write initial oid 44 2023-01-24T19:16:29.476 INFO:tasks.rados.rados.0.smithi161.stdout:29: finishing write tid 3 to smithi161112115-29 2023-01-24T19:16:29.477 INFO:tasks.rados.rados.0.smithi161.stdout:29: finishing write tid 4 to smithi161112115-29 2023-01-24T19:16:29.477 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 29 v 15 (ObjNum 28 snap 0 seq_num 28) dirty exists 2023-01-24T19:16:29.477 INFO:tasks.rados.rados.0.smithi161.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2023-01-24T19:16:29.477 INFO:tasks.rados.rados.0.smithi161.stdout:30: finishing write tid 1 to smithi161112115-30 2023-01-24T19:16:29.477 INFO:tasks.rados.rados.0.smithi161.stdout:30: finishing write tid 2 to smithi161112115-30 2023-01-24T19:16:29.477 INFO:tasks.rados.rados.0.smithi161.stdout:30: finishing write tid 3 to smithi161112115-30 2023-01-24T19:16:29.478 INFO:tasks.rados.rados.0.smithi161.stdout:30: finishing write tid 4 to smithi161112115-30 2023-01-24T19:16:29.478 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 30 v 18 (ObjNum 29 snap 0 seq_num 29) dirty exists 2023-01-24T19:16:29.478 INFO:tasks.rados.rados.0.smithi161.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2023-01-24T19:16:29.478 INFO:tasks.rados.rados.0.smithi161.stdout:31: finishing write tid 1 to smithi161112115-31 2023-01-24T19:16:29.478 INFO:tasks.rados.rados.0.smithi161.stdout:44: seq_num 43 ranges {722701=617472,1816608=788001} 2023-01-24T19:16:29.483 INFO:tasks.rados.rados.0.smithi161.stdout:44: writing smithi161112115-44 from 722701 to 1340173 tid 1 2023-01-24T19:16:29.491 INFO:tasks.rados.rados.0.smithi161.stdout:44: writing smithi161112115-44 from 1816608 to 2604609 tid 2 2023-01-24T19:16:29.491 INFO:tasks.rados.rados.0.smithi161.stdout:29: done (15 left) 2023-01-24T19:16:29.491 INFO:tasks.rados.rados.0.smithi161.stdout:30: done (14 left) 2023-01-24T19:16:29.491 INFO:tasks.rados.rados.0.smithi161.stdout:45: write initial oid 45 2023-01-24T19:16:29.491 INFO:tasks.rados.rados.0.smithi161.stdout:45: seq_num 44 ranges {739508=692721,1875090=648012} 2023-01-24T19:16:29.498 INFO:tasks.rados.rados.0.smithi161.stdout:45: writing smithi161112115-45 from 739508 to 1432229 tid 1 2023-01-24T19:16:29.505 INFO:tasks.rados.rados.0.smithi161.stdout:45: writing smithi161112115-45 from 1875090 to 2523102 tid 2 2023-01-24T19:16:29.505 INFO:tasks.rados.rados.0.smithi161.stdout:46: write initial oid 46 2023-01-24T19:16:29.505 INFO:tasks.rados.rados.0.smithi161.stdout:46: seq_num 45 ranges {756315=767970,1933572=791670} 2023-01-24T19:16:29.513 INFO:tasks.rados.rados.0.smithi161.stdout:46: writing smithi161112115-46 from 756315 to 1524285 tid 1 2023-01-24T19:16:29.523 INFO:tasks.rados.rados.0.smithi161.stdout:46: writing smithi161112115-46 from 1933572 to 2725242 tid 2 2023-01-24T19:16:29.523 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.523 INFO:tasks.rados.rados.0.smithi161.stdout:31: finishing write tid 2 to smithi161112115-31 2023-01-24T19:16:29.523 INFO:tasks.rados.rados.0.smithi161.stdout:31: finishing write tid 3 to smithi161112115-31 2023-01-24T19:16:29.524 INFO:tasks.rados.rados.0.smithi161.stdout:31: finishing write tid 4 to smithi161112115-31 2023-01-24T19:16:29.524 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 31 v 24 (ObjNum 30 snap 0 seq_num 30) dirty exists 2023-01-24T19:16:29.524 INFO:tasks.rados.rados.0.smithi161.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2023-01-24T19:16:29.524 INFO:tasks.rados.rados.0.smithi161.stdout:32: finishing write tid 1 to smithi161112115-32 2023-01-24T19:16:29.524 INFO:tasks.rados.rados.0.smithi161.stdout:32: finishing write tid 2 to smithi161112115-32 2023-01-24T19:16:29.525 INFO:tasks.rados.rados.0.smithi161.stdout:32: finishing write tid 3 to smithi161112115-32 2023-01-24T19:16:29.525 INFO:tasks.rados.rados.0.smithi161.stdout:32: finishing write tid 4 to smithi161112115-32 2023-01-24T19:16:29.525 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 32 v 21 (ObjNum 31 snap 0 seq_num 31) dirty exists 2023-01-24T19:16:29.525 INFO:tasks.rados.rados.0.smithi161.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2023-01-24T19:16:29.525 INFO:tasks.rados.rados.0.smithi161.stdout:33: finishing write tid 1 to smithi161112115-33 2023-01-24T19:16:29.526 INFO:tasks.rados.rados.0.smithi161.stdout:31: done (15 left) 2023-01-24T19:16:29.526 INFO:tasks.rados.rados.0.smithi161.stdout:32: done (14 left) 2023-01-24T19:16:29.526 INFO:tasks.rados.rados.0.smithi161.stdout:47: write initial oid 47 2023-01-24T19:16:29.526 INFO:tasks.rados.rados.0.smithi161.stdout:47: seq_num 46 ranges {773122=559572,1992054=651681} 2023-01-24T19:16:29.528 INFO:tasks.rados.rados.0.smithi161.stdout:47: writing smithi161112115-47 from 773122 to 1332694 tid 1 2023-01-24T19:16:29.535 INFO:tasks.rados.rados.0.smithi161.stdout:47: writing smithi161112115-47 from 1992054 to 2643735 tid 2 2023-01-24T19:16:29.536 INFO:tasks.rados.rados.0.smithi161.stdout:48: write initial oid 48 2023-01-24T19:16:29.536 INFO:tasks.rados.rados.0.smithi161.stdout:48: seq_num 47 ranges {789929=634821,2050536=739393} 2023-01-24T19:16:29.543 INFO:tasks.rados.rados.0.smithi161.stdout:48: writing smithi161112115-48 from 789929 to 1424750 tid 1 2023-01-24T19:16:29.552 INFO:tasks.rados.rados.0.smithi161.stdout:48: writing smithi161112115-48 from 2050536 to 2789929 tid 2 2023-01-24T19:16:29.552 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.552 INFO:tasks.rados.rados.0.smithi161.stdout:33: finishing write tid 2 to smithi161112115-33 2023-01-24T19:16:29.553 INFO:tasks.rados.rados.0.smithi161.stdout:33: finishing write tid 3 to smithi161112115-33 2023-01-24T19:16:29.553 INFO:tasks.rados.rados.0.smithi161.stdout:33: finishing write tid 4 to smithi161112115-33 2023-01-24T19:16:29.553 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 33 v 27 (ObjNum 32 snap 0 seq_num 32) dirty exists 2023-01-24T19:16:29.553 INFO:tasks.rados.rados.0.smithi161.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2023-01-24T19:16:29.553 INFO:tasks.rados.rados.0.smithi161.stdout:34: finishing write tid 1 to smithi161112115-34 2023-01-24T19:16:29.554 INFO:tasks.rados.rados.0.smithi161.stdout:33: done (15 left) 2023-01-24T19:16:29.554 INFO:tasks.rados.rados.0.smithi161.stdout:49: write initial oid 49 2023-01-24T19:16:29.554 INFO:tasks.rados.rados.0.smithi161.stdout:49: seq_num 48 ranges {406736=710070,1709018=655350} 2023-01-24T19:16:29.557 INFO:tasks.rados.rados.0.smithi161.stdout:49: writing smithi161112115-49 from 406736 to 1116806 tid 1 2023-01-24T19:16:29.565 INFO:tasks.rados.rados.0.smithi161.stdout:49: writing smithi161112115-49 from 1709018 to 2364368 tid 2 2023-01-24T19:16:29.566 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.566 INFO:tasks.rados.rados.0.smithi161.stdout:34: finishing write tid 2 to smithi161112115-34 2023-01-24T19:16:29.566 INFO:tasks.rados.rados.0.smithi161.stdout:34: finishing write tid 3 to smithi161112115-34 2023-01-24T19:16:29.566 INFO:tasks.rados.rados.0.smithi161.stdout:34: finishing write tid 4 to smithi161112115-34 2023-01-24T19:16:29.566 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 34 v 24 (ObjNum 33 snap 0 seq_num 33) dirty exists 2023-01-24T19:16:29.567 INFO:tasks.rados.rados.0.smithi161.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2023-01-24T19:16:29.567 INFO:tasks.rados.rados.0.smithi161.stdout:35: finishing write tid 1 to smithi161112115-35 2023-01-24T19:16:29.567 INFO:tasks.rados.rados.0.smithi161.stdout:34: done (15 left) 2023-01-24T19:16:29.567 INFO:tasks.rados.rados.0.smithi161.stdout:50: write initial oid 50 2023-01-24T19:16:29.567 INFO:tasks.rados.rados.0.smithi161.stdout:50: seq_num 49 ranges {423543=785319,1767500=799008} 2023-01-24T19:16:29.571 INFO:tasks.rados.rados.0.smithi161.stdout:50: writing smithi161112115-50 from 423543 to 1208862 tid 1 2023-01-24T19:16:29.580 INFO:tasks.rados.rados.0.smithi161.stdout:50: writing smithi161112115-50 from 1767500 to 2566508 tid 2 2023-01-24T19:16:29.580 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.581 INFO:tasks.rados.rados.0.smithi161.stdout:35: finishing write tid 2 to smithi161112115-35 2023-01-24T19:16:29.581 INFO:tasks.rados.rados.0.smithi161.stdout:35: finishing write tid 3 to smithi161112115-35 2023-01-24T19:16:29.581 INFO:tasks.rados.rados.0.smithi161.stdout:35: finishing write tid 4 to smithi161112115-35 2023-01-24T19:16:29.581 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 35 v 21 (ObjNum 34 snap 0 seq_num 34) dirty exists 2023-01-24T19:16:29.581 INFO:tasks.rados.rados.0.smithi161.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2023-01-24T19:16:29.582 INFO:tasks.rados.rados.0.smithi161.stdout:36: finishing write tid 1 to smithi161112115-36 2023-01-24T19:16:29.582 INFO:tasks.rados.rados.0.smithi161.stdout:35: done (15 left) 2023-01-24T19:16:29.582 INFO:tasks.rados.rados.0.smithi161.stdout:51: write initial oid 51 2023-01-24T19:16:29.582 INFO:tasks.rados.rados.0.smithi161.stdout:51: seq_num 50 ranges {440350=460568,1309629=542666,2323973=516377} 2023-01-24T19:16:29.584 INFO:tasks.rados.rados.0.smithi161.stdout:51: writing smithi161112115-51 from 440350 to 900918 tid 1 2023-01-24T19:16:29.590 INFO:tasks.rados.rados.0.smithi161.stdout:51: writing smithi161112115-51 from 1309629 to 1852295 tid 2 2023-01-24T19:16:29.596 INFO:tasks.rados.rados.0.smithi161.stdout:51: writing smithi161112115-51 from 2323973 to 2840350 tid 3 2023-01-24T19:16:29.596 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.597 INFO:tasks.rados.rados.0.smithi161.stdout:36: finishing write tid 2 to smithi161112115-36 2023-01-24T19:16:29.597 INFO:tasks.rados.rados.0.smithi161.stdout:36: finishing write tid 3 to smithi161112115-36 2023-01-24T19:16:29.597 INFO:tasks.rados.rados.0.smithi161.stdout:36: finishing write tid 4 to smithi161112115-36 2023-01-24T19:16:29.597 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 36 v 18 (ObjNum 35 snap 0 seq_num 35) dirty exists 2023-01-24T19:16:29.597 INFO:tasks.rados.rados.0.smithi161.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2023-01-24T19:16:29.598 INFO:tasks.rados.rados.0.smithi161.stdout:37: finishing write tid 1 to smithi161112115-37 2023-01-24T19:16:29.598 INFO:tasks.rados.rados.0.smithi161.stdout:37: finishing write tid 2 to smithi161112115-37 2023-01-24T19:16:29.598 INFO:tasks.rados.rados.0.smithi161.stdout:36: done (15 left) 2023-01-24T19:16:29.598 INFO:tasks.rados.rados.0.smithi161.stdout:52: write initial oid 52 2023-01-24T19:16:29.598 INFO:tasks.rados.rados.0.smithi161.stdout:52: seq_num 51 ranges {457157=535817,1768111=402677} 2023-01-24T19:16:29.601 INFO:tasks.rados.rados.0.smithi161.stdout:52: writing smithi161112115-52 from 457157 to 992974 tid 1 2023-01-24T19:16:29.608 INFO:tasks.rados.rados.0.smithi161.stdout:52: writing smithi161112115-52 from 1768111 to 2170788 tid 2 2023-01-24T19:16:29.608 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.609 INFO:tasks.rados.rados.0.smithi161.stdout:37: finishing write tid 3 to smithi161112115-37 2023-01-24T19:16:29.609 INFO:tasks.rados.rados.0.smithi161.stdout:37: finishing write tid 4 to smithi161112115-37 2023-01-24T19:16:29.609 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 37 v 24 (ObjNum 36 snap 0 seq_num 36) dirty exists 2023-01-24T19:16:29.609 INFO:tasks.rados.rados.0.smithi161.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2023-01-24T19:16:29.609 INFO:tasks.rados.rados.0.smithi161.stdout:38: finishing write tid 1 to smithi161112115-38 2023-01-24T19:16:29.610 INFO:tasks.rados.rados.0.smithi161.stdout:38: finishing write tid 2 to smithi161112115-38 2023-01-24T19:16:29.610 INFO:tasks.rados.rados.0.smithi161.stdout:38: finishing write tid 3 to smithi161112115-38 2023-01-24T19:16:29.610 INFO:tasks.rados.rados.0.smithi161.stdout:37: done (15 left) 2023-01-24T19:16:29.610 INFO:tasks.rados.rados.0.smithi161.stdout:53: write initial oid 53 2023-01-24T19:16:29.611 INFO:tasks.rados.rados.0.smithi161.stdout:53: seq_num 52 ranges {473964=611066,1826593=546335,2778819=95145} 2023-01-24T19:16:29.614 INFO:tasks.rados.rados.0.smithi161.stdout:53: writing smithi161112115-53 from 473964 to 1085030 tid 1 2023-01-24T19:16:29.621 INFO:tasks.rados.rados.0.smithi161.stdout:53: writing smithi161112115-53 from 1826593 to 2372928 tid 2 2023-01-24T19:16:29.625 INFO:tasks.rados.rados.0.smithi161.stdout:53: writing smithi161112115-53 from 2778819 to 2873964 tid 3 2023-01-24T19:16:29.625 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.625 INFO:tasks.rados.rados.0.smithi161.stdout:38: finishing write tid 4 to smithi161112115-38 2023-01-24T19:16:29.625 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 38 v 27 (ObjNum 37 snap 0 seq_num 37) dirty exists 2023-01-24T19:16:29.626 INFO:tasks.rados.rados.0.smithi161.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2023-01-24T19:16:29.626 INFO:tasks.rados.rados.0.smithi161.stdout:39: finishing write tid 1 to smithi161112115-39 2023-01-24T19:16:29.626 INFO:tasks.rados.rados.0.smithi161.stdout:38: done (15 left) 2023-01-24T19:16:29.626 INFO:tasks.rados.rados.0.smithi161.stdout:54: write initial oid 54 2023-01-24T19:16:29.626 INFO:tasks.rados.rados.0.smithi161.stdout:39: finishing write tid 2 to smithi161112115-39 2023-01-24T19:16:29.627 INFO:tasks.rados.rados.0.smithi161.stdout:39: finishing write tid 3 to smithi161112115-39 2023-01-24T19:16:29.627 INFO:tasks.rados.rados.0.smithi161.stdout:39: finishing write tid 4 to smithi161112115-39 2023-01-24T19:16:29.627 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 39 v 12 (ObjNum 38 snap 0 seq_num 38) dirty exists 2023-01-24T19:16:29.627 INFO:tasks.rados.rados.0.smithi161.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2023-01-24T19:16:29.627 INFO:tasks.rados.rados.0.smithi161.stdout:40: finishing write tid 1 to smithi161112115-40 2023-01-24T19:16:29.628 INFO:tasks.rados.rados.0.smithi161.stdout:54: seq_num 53 ranges {490771=686315,1885075=406346} 2023-01-24T19:16:29.631 INFO:tasks.rados.rados.0.smithi161.stdout:54: writing smithi161112115-54 from 490771 to 1177086 tid 1 2023-01-24T19:16:29.637 INFO:tasks.rados.rados.0.smithi161.stdout:54: writing smithi161112115-54 from 1885075 to 2291421 tid 2 2023-01-24T19:16:29.637 INFO:tasks.rados.rados.0.smithi161.stdout:39: done (15 left) 2023-01-24T19:16:29.637 INFO:tasks.rados.rados.0.smithi161.stdout:55: write initial oid 55 2023-01-24T19:16:29.638 INFO:tasks.rados.rados.0.smithi161.stdout:55: seq_num 54 ranges {507578=477917,1543557=550004,2833665=73913} 2023-01-24T19:16:29.642 INFO:tasks.rados.rados.0.smithi161.stdout:55: writing smithi161112115-55 from 507578 to 985495 tid 1 2023-01-24T19:16:29.649 INFO:tasks.rados.rados.0.smithi161.stdout:55: writing smithi161112115-55 from 1543557 to 2093561 tid 2 2023-01-24T19:16:29.654 INFO:tasks.rados.rados.0.smithi161.stdout:55: writing smithi161112115-55 from 2833665 to 2907578 tid 3 2023-01-24T19:16:29.654 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.654 INFO:tasks.rados.rados.0.smithi161.stdout:40: finishing write tid 2 to smithi161112115-40 2023-01-24T19:16:29.654 INFO:tasks.rados.rados.0.smithi161.stdout:40: finishing write tid 3 to smithi161112115-40 2023-01-24T19:16:29.655 INFO:tasks.rados.rados.0.smithi161.stdout:40: finishing write tid 4 to smithi161112115-40 2023-01-24T19:16:29.655 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 40 v 27 (ObjNum 39 snap 0 seq_num 39) dirty exists 2023-01-24T19:16:29.655 INFO:tasks.rados.rados.0.smithi161.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2023-01-24T19:16:29.655 INFO:tasks.rados.rados.0.smithi161.stdout:41: finishing write tid 1 to smithi161112115-41 2023-01-24T19:16:29.655 INFO:tasks.rados.rados.0.smithi161.stdout:41: finishing write tid 2 to smithi161112115-41 2023-01-24T19:16:29.656 INFO:tasks.rados.rados.0.smithi161.stdout:40: done (15 left) 2023-01-24T19:16:29.656 INFO:tasks.rados.rados.0.smithi161.stdout:56: write initial oid 56 2023-01-24T19:16:29.656 INFO:tasks.rados.rados.0.smithi161.stdout:56: seq_num 55 ranges {524385=553166,1602039=410015,2577441=346944} 2023-01-24T19:16:29.659 INFO:tasks.rados.rados.0.smithi161.stdout:56: writing smithi161112115-56 from 524385 to 1077551 tid 1 2023-01-24T19:16:29.664 INFO:tasks.rados.rados.0.smithi161.stdout:56: writing smithi161112115-56 from 1602039 to 2012054 tid 2 2023-01-24T19:16:29.670 INFO:tasks.rados.rados.0.smithi161.stdout:56: writing smithi161112115-56 from 2577441 to 2924385 tid 3 2023-01-24T19:16:29.670 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.671 INFO:tasks.rados.rados.0.smithi161.stdout:41: finishing write tid 3 to smithi161112115-41 2023-01-24T19:16:29.671 INFO:tasks.rados.rados.0.smithi161.stdout:41: finishing write tid 4 to smithi161112115-41 2023-01-24T19:16:29.671 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 41 v 30 (ObjNum 40 snap 0 seq_num 40) dirty exists 2023-01-24T19:16:29.671 INFO:tasks.rados.rados.0.smithi161.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2023-01-24T19:16:29.671 INFO:tasks.rados.rados.0.smithi161.stdout:42: finishing write tid 1 to smithi161112115-42 2023-01-24T19:16:29.672 INFO:tasks.rados.rados.0.smithi161.stdout:42: finishing write tid 2 to smithi161112115-42 2023-01-24T19:16:29.672 INFO:tasks.rados.rados.0.smithi161.stdout:42: finishing write tid 3 to smithi161112115-42 2023-01-24T19:16:29.672 INFO:tasks.rados.rados.0.smithi161.stdout:42: finishing write tid 4 to smithi161112115-42 2023-01-24T19:16:29.672 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 42 v 21 (ObjNum 41 snap 0 seq_num 41) dirty exists 2023-01-24T19:16:29.672 INFO:tasks.rados.rados.0.smithi161.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2023-01-24T19:16:29.673 INFO:tasks.rados.rados.0.smithi161.stdout:43: finishing write tid 1 to smithi161112115-43 2023-01-24T19:16:29.673 INFO:tasks.rados.rados.0.smithi161.stdout:41: done (15 left) 2023-01-24T19:16:29.673 INFO:tasks.rados.rados.0.smithi161.stdout:42: done (14 left) 2023-01-24T19:16:29.673 INFO:tasks.rados.rados.0.smithi161.stdout:57: write initial oid 57 2023-01-24T19:16:29.674 INFO:tasks.rados.rados.0.smithi161.stdout:57: seq_num 56 ranges {541192=628415,1660521=553673,2888511=52681} 2023-01-24T19:16:29.676 INFO:tasks.rados.rados.0.smithi161.stdout:57: writing smithi161112115-57 from 541192 to 1169607 tid 1 2023-01-24T19:16:29.682 INFO:tasks.rados.rados.0.smithi161.stdout:57: writing smithi161112115-57 from 1660521 to 2214194 tid 2 2023-01-24T19:16:29.686 INFO:tasks.rados.rados.0.smithi161.stdout:57: writing smithi161112115-57 from 2888511 to 2941192 tid 3 2023-01-24T19:16:29.686 INFO:tasks.rados.rados.0.smithi161.stdout:58: write initial oid 58 2023-01-24T19:16:29.686 INFO:tasks.rados.rados.0.smithi161.stdout:58: seq_num 57 ranges {557999=703664,1719003=413684,2632287=325712} 2023-01-24T19:16:29.693 INFO:tasks.rados.rados.0.smithi161.stdout:58: writing smithi161112115-58 from 557999 to 1261663 tid 1 2023-01-24T19:16:29.698 INFO:tasks.rados.rados.0.smithi161.stdout:58: writing smithi161112115-58 from 1719003 to 2132687 tid 2 2023-01-24T19:16:29.703 INFO:tasks.rados.rados.0.smithi161.stdout:58: writing smithi161112115-58 from 2632287 to 2957999 tid 3 2023-01-24T19:16:29.703 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.703 INFO:tasks.rados.rados.0.smithi161.stdout:43: finishing write tid 2 to smithi161112115-43 2023-01-24T19:16:29.703 INFO:tasks.rados.rados.0.smithi161.stdout:43: finishing write tid 3 to smithi161112115-43 2023-01-24T19:16:29.704 INFO:tasks.rados.rados.0.smithi161.stdout:43: finishing write tid 4 to smithi161112115-43 2023-01-24T19:16:29.704 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 43 v 30 (ObjNum 42 snap 0 seq_num 42) dirty exists 2023-01-24T19:16:29.704 INFO:tasks.rados.rados.0.smithi161.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2023-01-24T19:16:29.704 INFO:tasks.rados.rados.0.smithi161.stdout:44: finishing write tid 1 to smithi161112115-44 2023-01-24T19:16:29.704 INFO:tasks.rados.rados.0.smithi161.stdout:44: finishing write tid 2 to smithi161112115-44 2023-01-24T19:16:29.705 INFO:tasks.rados.rados.0.smithi161.stdout:43: done (15 left) 2023-01-24T19:16:29.705 INFO:tasks.rados.rados.0.smithi161.stdout:59: write initial oid 59 2023-01-24T19:16:29.705 INFO:tasks.rados.rados.0.smithi161.stdout:59: seq_num 58 ranges {574806=778913,2061132=557342} 2023-01-24T19:16:29.710 INFO:tasks.rados.rados.0.smithi161.stdout:59: writing smithi161112115-59 from 574806 to 1353719 tid 1 2023-01-24T19:16:29.718 INFO:tasks.rados.rados.0.smithi161.stdout:59: writing smithi161112115-59 from 2061132 to 2618474 tid 2 2023-01-24T19:16:29.718 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.718 INFO:tasks.rados.rados.0.smithi161.stdout:44: finishing write tid 3 to smithi161112115-44 2023-01-24T19:16:29.718 INFO:tasks.rados.rados.0.smithi161.stdout:44: finishing write tid 4 to smithi161112115-44 2023-01-24T19:16:29.719 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 44 v 30 (ObjNum 43 snap 0 seq_num 43) dirty exists 2023-01-24T19:16:29.719 INFO:tasks.rados.rados.0.smithi161.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2023-01-24T19:16:29.719 INFO:tasks.rados.rados.0.smithi161.stdout:45: finishing write tid 1 to smithi161112115-45 2023-01-24T19:16:29.719 INFO:tasks.rados.rados.0.smithi161.stdout:45: finishing write tid 2 to smithi161112115-45 2023-01-24T19:16:29.719 INFO:tasks.rados.rados.0.smithi161.stdout:45: finishing write tid 3 to smithi161112115-45 2023-01-24T19:16:29.720 INFO:tasks.rados.rados.0.smithi161.stdout:45: finishing write tid 4 to smithi161112115-45 2023-01-24T19:16:29.720 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 45 v 24 (ObjNum 44 snap 0 seq_num 44) dirty exists 2023-01-24T19:16:29.720 INFO:tasks.rados.rados.0.smithi161.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2023-01-24T19:16:29.720 INFO:tasks.rados.rados.0.smithi161.stdout:44: done (15 left) 2023-01-24T19:16:29.721 INFO:tasks.rados.rados.0.smithi161.stdout:45: done (14 left) 2023-01-24T19:16:29.721 INFO:tasks.rados.rados.0.smithi161.stdout:60: write initial oid 60 2023-01-24T19:16:29.721 INFO:tasks.rados.rados.0.smithi161.stdout:60: seq_num 59 ranges {591613=454162,1719614=417353,2570780=420833} 2023-01-24T19:16:29.723 INFO:tasks.rados.rados.0.smithi161.stdout:60: writing smithi161112115-60 from 591613 to 1045775 tid 1 2023-01-24T19:16:29.729 INFO:tasks.rados.rados.0.smithi161.stdout:60: writing smithi161112115-60 from 1719614 to 2136967 tid 2 2023-01-24T19:16:29.734 INFO:tasks.rados.rados.0.smithi161.stdout:60: writing smithi161112115-60 from 2570780 to 2991613 tid 3 2023-01-24T19:16:29.734 INFO:tasks.rados.rados.0.smithi161.stdout:61: write initial oid 61 2023-01-24T19:16:29.734 INFO:tasks.rados.rados.0.smithi161.stdout:61: seq_num 60 ranges {608420=529411,1778096=561011,2881850=126570} 2023-01-24T19:16:29.740 INFO:tasks.rados.rados.0.smithi161.stdout:61: writing smithi161112115-61 from 608420 to 1137831 tid 1 2023-01-24T19:16:29.747 INFO:tasks.rados.rados.0.smithi161.stdout:61: writing smithi161112115-61 from 1778096 to 2339107 tid 2 2023-01-24T19:16:29.751 INFO:tasks.rados.rados.0.smithi161.stdout:61: writing smithi161112115-61 from 2881850 to 3008420 tid 3 2023-01-24T19:16:29.751 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.751 INFO:tasks.rados.rados.0.smithi161.stdout:46: finishing write tid 1 to smithi161112115-46 2023-01-24T19:16:29.752 INFO:tasks.rados.rados.0.smithi161.stdout:46: finishing write tid 2 to smithi161112115-46 2023-01-24T19:16:29.752 INFO:tasks.rados.rados.0.smithi161.stdout:46: finishing write tid 3 to smithi161112115-46 2023-01-24T19:16:29.752 INFO:tasks.rados.rados.0.smithi161.stdout:46: finishing write tid 4 to smithi161112115-46 2023-01-24T19:16:29.752 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 46 v 33 (ObjNum 45 snap 0 seq_num 45) dirty exists 2023-01-24T19:16:29.752 INFO:tasks.rados.rados.0.smithi161.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2023-01-24T19:16:29.753 INFO:tasks.rados.rados.0.smithi161.stdout:47: finishing write tid 1 to smithi161112115-47 2023-01-24T19:16:29.753 INFO:tasks.rados.rados.0.smithi161.stdout:47: finishing write tid 2 to smithi161112115-47 2023-01-24T19:16:29.753 INFO:tasks.rados.rados.0.smithi161.stdout:47: finishing write tid 3 to smithi161112115-47 2023-01-24T19:16:29.753 INFO:tasks.rados.rados.0.smithi161.stdout:46: done (15 left) 2023-01-24T19:16:29.753 INFO:tasks.rados.rados.0.smithi161.stdout:62: write initial oid 62 2023-01-24T19:16:29.754 INFO:tasks.rados.rados.0.smithi161.stdout:62: seq_num 61 ranges {625227=721013,1952931=704669} 2023-01-24T19:16:29.758 INFO:tasks.rados.rados.0.smithi161.stdout:62: writing smithi161112115-62 from 625227 to 1346240 tid 1 2023-01-24T19:16:29.766 INFO:tasks.rados.rados.0.smithi161.stdout:62: writing smithi161112115-62 from 1952931 to 2657600 tid 2 2023-01-24T19:16:29.766 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.766 INFO:tasks.rados.rados.0.smithi161.stdout:47: finishing write tid 4 to smithi161112115-47 2023-01-24T19:16:29.767 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 47 v 33 (ObjNum 46 snap 0 seq_num 46) dirty exists 2023-01-24T19:16:29.767 INFO:tasks.rados.rados.0.smithi161.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2023-01-24T19:16:29.767 INFO:tasks.rados.rados.0.smithi161.stdout:48: finishing write tid 1 to smithi161112115-48 2023-01-24T19:16:29.767 INFO:tasks.rados.rados.0.smithi161.stdout:48: finishing write tid 2 to smithi161112115-48 2023-01-24T19:16:29.768 INFO:tasks.rados.rados.0.smithi161.stdout:48: finishing write tid 3 to smithi161112115-48 2023-01-24T19:16:29.768 INFO:tasks.rados.rados.0.smithi161.stdout:48: finishing write tid 4 to smithi161112115-48 2023-01-24T19:16:29.768 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 48 v 27 (ObjNum 47 snap 0 seq_num 47) dirty exists 2023-01-24T19:16:29.768 INFO:tasks.rados.rados.0.smithi161.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2023-01-24T19:16:29.768 INFO:tasks.rados.rados.0.smithi161.stdout:49: finishing write tid 1 to smithi161112115-49 2023-01-24T19:16:29.769 INFO:tasks.rados.rados.0.smithi161.stdout:47: done (15 left) 2023-01-24T19:16:29.769 INFO:tasks.rados.rados.0.smithi161.stdout:48: done (14 left) 2023-01-24T19:16:29.769 INFO:tasks.rados.rados.0.smithi161.stdout:63: write initial oid 63 2023-01-24T19:16:29.769 INFO:tasks.rados.rados.0.smithi161.stdout:49: finishing write tid 2 to smithi161112115-49 2023-01-24T19:16:29.770 INFO:tasks.rados.rados.0.smithi161.stdout:49: finishing write tid 3 to smithi161112115-49 2023-01-24T19:16:29.770 INFO:tasks.rados.rados.0.smithi161.stdout:63: seq_num 62 ranges {642034=796262,1895060=564680} 2023-01-24T19:16:29.775 INFO:tasks.rados.rados.0.smithi161.stdout:63: writing smithi161112115-63 from 642034 to 1438296 tid 1 2023-01-24T19:16:29.780 INFO:tasks.rados.rados.0.smithi161.stdout:63: writing smithi161112115-63 from 1895060 to 2459740 tid 2 2023-01-24T19:16:29.780 INFO:tasks.rados.rados.0.smithi161.stdout:64: write initial oid 64 2023-01-24T19:16:29.780 INFO:tasks.rados.rados.0.smithi161.stdout:64: seq_num 63 ranges {658841=471511,1553542=708338,2964119=94722} 2023-01-24T19:16:29.786 INFO:tasks.rados.rados.0.smithi161.stdout:64: writing smithi161112115-64 from 658841 to 1130352 tid 1 2023-01-24T19:16:29.792 INFO:tasks.rados.rados.0.smithi161.stdout:64: writing smithi161112115-64 from 1553542 to 2261880 tid 2 2023-01-24T19:16:29.798 INFO:tasks.rados.rados.0.smithi161.stdout:64: writing smithi161112115-64 from 2964119 to 3058841 tid 3 2023-01-24T19:16:29.798 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.798 INFO:tasks.rados.rados.0.smithi161.stdout:49: finishing write tid 4 to smithi161112115-49 2023-01-24T19:16:29.798 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 49 v 13 (ObjNum 48 snap 0 seq_num 48) dirty exists 2023-01-24T19:16:29.798 INFO:tasks.rados.rados.0.smithi161.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2023-01-24T19:16:29.799 INFO:tasks.rados.rados.0.smithi161.stdout:50: finishing write tid 1 to smithi161112115-50 2023-01-24T19:16:29.799 INFO:tasks.rados.rados.0.smithi161.stdout:50: finishing write tid 2 to smithi161112115-50 2023-01-24T19:16:29.799 INFO:tasks.rados.rados.0.smithi161.stdout:50: finishing write tid 3 to smithi161112115-50 2023-01-24T19:16:29.799 INFO:tasks.rados.rados.0.smithi161.stdout:50: finishing write tid 4 to smithi161112115-50 2023-01-24T19:16:29.799 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 50 v 16 (ObjNum 49 snap 0 seq_num 49) dirty exists 2023-01-24T19:16:29.799 INFO:tasks.rados.rados.0.smithi161.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2023-01-24T19:16:29.800 INFO:tasks.rados.rados.0.smithi161.stdout:51: finishing write tid 1 to smithi161112115-51 2023-01-24T19:16:29.800 INFO:tasks.rados.rados.0.smithi161.stdout:51: finishing write tid 2 to smithi161112115-51 2023-01-24T19:16:29.800 INFO:tasks.rados.rados.0.smithi161.stdout:51: finishing write tid 3 to smithi161112115-51 2023-01-24T19:16:29.800 INFO:tasks.rados.rados.0.smithi161.stdout:51: finishing write tid 4 to smithi161112115-51 2023-01-24T19:16:29.800 INFO:tasks.rados.rados.0.smithi161.stdout:51: finishing write tid 5 to smithi161112115-51 2023-01-24T19:16:29.801 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 51 v 37 (ObjNum 50 snap 0 seq_num 50) dirty exists 2023-01-24T19:16:29.801 INFO:tasks.rados.rados.0.smithi161.stdout:51: left oid 51 (ObjNum 50 snap 0 seq_num 50) 2023-01-24T19:16:29.801 INFO:tasks.rados.rados.0.smithi161.stdout:52: finishing write tid 1 to smithi161112115-52 2023-01-24T19:16:29.801 INFO:tasks.rados.rados.0.smithi161.stdout:52: finishing write tid 2 to smithi161112115-52 2023-01-24T19:16:29.801 INFO:tasks.rados.rados.0.smithi161.stdout:52: finishing write tid 3 to smithi161112115-52 2023-01-24T19:16:29.802 INFO:tasks.rados.rados.0.smithi161.stdout:52: finishing write tid 4 to smithi161112115-52 2023-01-24T19:16:29.802 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 52 v 36 (ObjNum 51 snap 0 seq_num 51) dirty exists 2023-01-24T19:16:29.802 INFO:tasks.rados.rados.0.smithi161.stdout:52: left oid 52 (ObjNum 51 snap 0 seq_num 51) 2023-01-24T19:16:29.802 INFO:tasks.rados.rados.0.smithi161.stdout:53: finishing write tid 1 to smithi161112115-53 2023-01-24T19:16:29.802 INFO:tasks.rados.rados.0.smithi161.stdout:53: finishing write tid 2 to smithi161112115-53 2023-01-24T19:16:29.803 INFO:tasks.rados.rados.0.smithi161.stdout:53: finishing write tid 3 to smithi161112115-53 2023-01-24T19:16:29.803 INFO:tasks.rados.rados.0.smithi161.stdout:53: finishing write tid 4 to smithi161112115-53 2023-01-24T19:16:29.803 INFO:tasks.rados.rados.0.smithi161.stdout:53: finishing write tid 5 to smithi161112115-53 2023-01-24T19:16:29.803 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 53 v 40 (ObjNum 52 snap 0 seq_num 52) dirty exists 2023-01-24T19:16:29.803 INFO:tasks.rados.rados.0.smithi161.stdout:53: left oid 53 (ObjNum 52 snap 0 seq_num 52) 2023-01-24T19:16:29.803 INFO:tasks.rados.rados.0.smithi161.stdout:54: finishing write tid 1 to smithi161112115-54 2023-01-24T19:16:29.804 INFO:tasks.rados.rados.0.smithi161.stdout:54: finishing write tid 2 to smithi161112115-54 2023-01-24T19:16:29.804 INFO:tasks.rados.rados.0.smithi161.stdout:54: finishing write tid 3 to smithi161112115-54 2023-01-24T19:16:29.804 INFO:tasks.rados.rados.0.smithi161.stdout:54: finishing write tid 4 to smithi161112115-54 2023-01-24T19:16:29.804 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 54 v 43 (ObjNum 53 snap 0 seq_num 53) dirty exists 2023-01-24T19:16:29.804 INFO:tasks.rados.rados.0.smithi161.stdout:54: left oid 54 (ObjNum 53 snap 0 seq_num 53) 2023-01-24T19:16:29.805 INFO:tasks.rados.rados.0.smithi161.stdout:55: finishing write tid 1 to smithi161112115-55 2023-01-24T19:16:29.805 INFO:tasks.rados.rados.0.smithi161.stdout:55: finishing write tid 2 to smithi161112115-55 2023-01-24T19:16:29.805 INFO:tasks.rados.rados.0.smithi161.stdout:55: finishing write tid 3 to smithi161112115-55 2023-01-24T19:16:29.805 INFO:tasks.rados.rados.0.smithi161.stdout:55: finishing write tid 4 to smithi161112115-55 2023-01-24T19:16:29.805 INFO:tasks.rados.rados.0.smithi161.stdout:55: finishing write tid 5 to smithi161112115-55 2023-01-24T19:16:29.806 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 55 v 21 (ObjNum 54 snap 0 seq_num 54) dirty exists 2023-01-24T19:16:29.806 INFO:tasks.rados.rados.0.smithi161.stdout:55: left oid 55 (ObjNum 54 snap 0 seq_num 54) 2023-01-24T19:16:29.806 INFO:tasks.rados.rados.0.smithi161.stdout:56: finishing write tid 1 to smithi161112115-56 2023-01-24T19:16:29.806 INFO:tasks.rados.rados.0.smithi161.stdout:56: finishing write tid 2 to smithi161112115-56 2023-01-24T19:16:29.806 INFO:tasks.rados.rados.0.smithi161.stdout:49: done (15 left) 2023-01-24T19:16:29.807 INFO:tasks.rados.rados.0.smithi161.stdout:50: done (14 left) 2023-01-24T19:16:29.807 INFO:tasks.rados.rados.0.smithi161.stdout:51: done (13 left) 2023-01-24T19:16:29.807 INFO:tasks.rados.rados.0.smithi161.stdout:52: done (12 left) 2023-01-24T19:16:29.807 INFO:tasks.rados.rados.0.smithi161.stdout:53: done (11 left) 2023-01-24T19:16:29.807 INFO:tasks.rados.rados.0.smithi161.stdout:54: done (10 left) 2023-01-24T19:16:29.808 INFO:tasks.rados.rados.0.smithi161.stdout:55: done (9 left) 2023-01-24T19:16:29.808 INFO:tasks.rados.rados.0.smithi161.stdout:65: write initial oid 65 2023-01-24T19:16:29.808 INFO:tasks.rados.rados.0.smithi161.stdout:65: seq_num 64 ranges {675648=546760,2012024=568349} 2023-01-24T19:16:29.808 INFO:tasks.rados.rados.0.smithi161.stdout:65: writing smithi161112115-65 from 675648 to 1222408 tid 1 2023-01-24T19:16:29.811 INFO:tasks.rados.rados.0.smithi161.stdout:65: writing smithi161112115-65 from 2012024 to 2580373 tid 2 2023-01-24T19:16:29.811 INFO:tasks.rados.rados.0.smithi161.stdout:66: write initial oid 66 2023-01-24T19:16:29.811 INFO:tasks.rados.rados.0.smithi161.stdout:66: seq_num 65 ranges {692455=622009,2070506=712007} 2023-01-24T19:16:29.818 INFO:tasks.rados.rados.0.smithi161.stdout:66: writing smithi161112115-66 from 692455 to 1314464 tid 1 2023-01-24T19:16:29.826 INFO:tasks.rados.rados.0.smithi161.stdout:66: writing smithi161112115-66 from 2070506 to 2782513 tid 2 2023-01-24T19:16:29.826 INFO:tasks.rados.rados.0.smithi161.stdout:67: write initial oid 67 2023-01-24T19:16:29.826 INFO:tasks.rados.rados.0.smithi161.stdout:67: seq_num 66 ranges {709262=697258,2012635=572018,3046388=62874} 2023-01-24T19:16:29.833 INFO:tasks.rados.rados.0.smithi161.stdout:67: writing smithi161112115-67 from 709262 to 1406520 tid 1 2023-01-24T19:16:29.840 INFO:tasks.rados.rados.0.smithi161.stdout:67: writing smithi161112115-67 from 2012635 to 2584653 tid 2 2023-01-24T19:16:29.843 INFO:tasks.rados.rados.0.smithi161.stdout:67: writing smithi161112115-67 from 3046388 to 3109262 tid 3 2023-01-24T19:16:29.843 INFO:tasks.rados.rados.0.smithi161.stdout:68: write initial oid 68 2023-01-24T19:16:29.844 INFO:tasks.rados.rados.0.smithi161.stdout:68: seq_num 67 ranges {726069=772507,2071117=715676} 2023-01-24T19:16:29.851 INFO:tasks.rados.rados.0.smithi161.stdout:68: writing smithi161112115-68 from 726069 to 1498576 tid 1 2023-01-24T19:16:29.859 INFO:tasks.rados.rados.0.smithi161.stdout:68: writing smithi161112115-68 from 2071117 to 2786793 tid 2 2023-01-24T19:16:29.859 INFO:tasks.rados.rados.0.smithi161.stdout:69: write initial oid 69 2023-01-24T19:16:29.859 INFO:tasks.rados.rados.0.smithi161.stdout:69: seq_num 68 ranges {742876=447756,1729599=575687,3101234=41642} 2023-01-24T19:16:29.865 INFO:tasks.rados.rados.0.smithi161.stdout:69: writing smithi161112115-69 from 742876 to 1190632 tid 1 2023-01-24T19:16:29.872 INFO:tasks.rados.rados.0.smithi161.stdout:69: writing smithi161112115-69 from 1729599 to 2305286 tid 2 2023-01-24T19:16:29.876 INFO:tasks.rados.rados.0.smithi161.stdout:69: writing smithi161112115-69 from 3101234 to 3142876 tid 3 2023-01-24T19:16:29.877 INFO:tasks.rados.rados.0.smithi161.stdout:70: write initial oid 70 2023-01-24T19:16:29.877 INFO:tasks.rados.rados.0.smithi161.stdout:70: seq_num 69 ranges {759683=639358,1904434=719345,3128657=31026} 2023-01-24T19:16:29.884 INFO:tasks.rados.rados.0.smithi161.stdout:70: writing smithi161112115-70 from 759683 to 1399041 tid 1 2023-01-24T19:16:29.891 INFO:tasks.rados.rados.0.smithi161.stdout:70: writing smithi161112115-70 from 1904434 to 2623779 tid 2 2023-01-24T19:16:29.894 INFO:tasks.rados.rados.0.smithi161.stdout:70: writing smithi161112115-70 from 3128657 to 3159683 tid 3 2023-01-24T19:16:29.895 INFO:tasks.rados.rados.0.smithi161.stdout:71: write initial oid 71 2023-01-24T19:16:29.895 INFO:tasks.rados.rados.0.smithi161.stdout:71: seq_num 70 ranges {776490=714607,2246563=579356} 2023-01-24T19:16:29.902 INFO:tasks.rados.rados.0.smithi161.stdout:71: writing smithi161112115-71 from 776490 to 1491097 tid 1 2023-01-24T19:16:29.911 INFO:tasks.rados.rados.0.smithi161.stdout:71: writing smithi161112115-71 from 2246563 to 2825919 tid 2 2023-01-24T19:16:29.911 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.911 INFO:tasks.rados.rados.0.smithi161.stdout:56: finishing write tid 3 to smithi161112115-56 2023-01-24T19:16:29.911 INFO:tasks.rados.rados.0.smithi161.stdout:56: finishing write tid 4 to smithi161112115-56 2023-01-24T19:16:29.911 INFO:tasks.rados.rados.0.smithi161.stdout:56: finishing write tid 5 to smithi161112115-56 2023-01-24T19:16:29.912 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 56 v 25 (ObjNum 55 snap 0 seq_num 55) dirty exists 2023-01-24T19:16:29.912 INFO:tasks.rados.rados.0.smithi161.stdout:56: left oid 56 (ObjNum 55 snap 0 seq_num 55) 2023-01-24T19:16:29.912 INFO:tasks.rados.rados.0.smithi161.stdout:57: finishing write tid 1 to smithi161112115-57 2023-01-24T19:16:29.912 INFO:tasks.rados.rados.0.smithi161.stdout:57: finishing write tid 2 to smithi161112115-57 2023-01-24T19:16:29.912 INFO:tasks.rados.rados.0.smithi161.stdout:57: finishing write tid 3 to smithi161112115-57 2023-01-24T19:16:29.913 INFO:tasks.rados.rados.0.smithi161.stdout:57: finishing write tid 4 to smithi161112115-57 2023-01-24T19:16:29.913 INFO:tasks.rados.rados.0.smithi161.stdout:57: finishing write tid 5 to smithi161112115-57 2023-01-24T19:16:29.913 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 57 v 20 (ObjNum 56 snap 0 seq_num 56) dirty exists 2023-01-24T19:16:29.913 INFO:tasks.rados.rados.0.smithi161.stdout:57: left oid 57 (ObjNum 56 snap 0 seq_num 56) 2023-01-24T19:16:29.913 INFO:tasks.rados.rados.0.smithi161.stdout:58: finishing write tid 1 to smithi161112115-58 2023-01-24T19:16:29.914 INFO:tasks.rados.rados.0.smithi161.stdout:58: finishing write tid 2 to smithi161112115-58 2023-01-24T19:16:29.914 INFO:tasks.rados.rados.0.smithi161.stdout:58: finishing write tid 3 to smithi161112115-58 2023-01-24T19:16:29.914 INFO:tasks.rados.rados.0.smithi161.stdout:58: finishing write tid 4 to smithi161112115-58 2023-01-24T19:16:29.914 INFO:tasks.rados.rados.0.smithi161.stdout:58: finishing write tid 5 to smithi161112115-58 2023-01-24T19:16:29.914 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 58 v 41 (ObjNum 57 snap 0 seq_num 57) dirty exists 2023-01-24T19:16:29.915 INFO:tasks.rados.rados.0.smithi161.stdout:58: left oid 58 (ObjNum 57 snap 0 seq_num 57) 2023-01-24T19:16:29.915 INFO:tasks.rados.rados.0.smithi161.stdout:59: finishing write tid 1 to smithi161112115-59 2023-01-24T19:16:29.915 INFO:tasks.rados.rados.0.smithi161.stdout:59: finishing write tid 2 to smithi161112115-59 2023-01-24T19:16:29.915 INFO:tasks.rados.rados.0.smithi161.stdout:59: finishing write tid 3 to smithi161112115-59 2023-01-24T19:16:29.915 INFO:tasks.rados.rados.0.smithi161.stdout:59: finishing write tid 4 to smithi161112115-59 2023-01-24T19:16:29.916 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 59 v 12 (ObjNum 58 snap 0 seq_num 58) dirty exists 2023-01-24T19:16:29.916 INFO:tasks.rados.rados.0.smithi161.stdout:59: left oid 59 (ObjNum 58 snap 0 seq_num 58) 2023-01-24T19:16:29.916 INFO:tasks.rados.rados.0.smithi161.stdout:56: done (15 left) 2023-01-24T19:16:29.916 INFO:tasks.rados.rados.0.smithi161.stdout:57: done (14 left) 2023-01-24T19:16:29.916 INFO:tasks.rados.rados.0.smithi161.stdout:58: done (13 left) 2023-01-24T19:16:29.917 INFO:tasks.rados.rados.0.smithi161.stdout:59: done (12 left) 2023-01-24T19:16:29.917 INFO:tasks.rados.rados.0.smithi161.stdout:72: write initial oid 72 2023-01-24T19:16:29.917 INFO:tasks.rados.rados.0.smithi161.stdout:72: seq_num 71 ranges {793297=789856,2305045=723014} 2023-01-24T19:16:29.919 INFO:tasks.rados.rados.0.smithi161.stdout:72: writing smithi161112115-72 from 793297 to 1583153 tid 1 2023-01-24T19:16:29.927 INFO:tasks.rados.rados.0.smithi161.stdout:72: writing smithi161112115-72 from 2305045 to 3028059 tid 2 2023-01-24T19:16:29.928 INFO:tasks.rados.rados.0.smithi161.stdout:73: write initial oid 73 2023-01-24T19:16:29.928 INFO:tasks.rados.rados.0.smithi161.stdout:73: seq_num 72 ranges {410104=465105,1563527=583025,2810926=399178} 2023-01-24T19:16:29.932 INFO:tasks.rados.rados.0.smithi161.stdout:73: writing smithi161112115-73 from 410104 to 875209 tid 1 2023-01-24T19:16:29.939 INFO:tasks.rados.rados.0.smithi161.stdout:73: writing smithi161112115-73 from 1563527 to 2146552 tid 2 2023-01-24T19:16:29.945 INFO:tasks.rados.rados.0.smithi161.stdout:73: writing smithi161112115-73 from 2810926 to 3210104 tid 3 2023-01-24T19:16:29.946 INFO:tasks.rados.rados.0.smithi161.stdout:74: write initial oid 74 2023-01-24T19:16:29.946 INFO:tasks.rados.rados.0.smithi161.stdout:74: seq_num 73 ranges {426911=540354,1622009=726683,3121996=104915} 2023-01-24T19:16:29.951 INFO:tasks.rados.rados.0.smithi161.stdout:74: writing smithi161112115-74 from 426911 to 967265 tid 1 2023-01-24T19:16:29.958 INFO:tasks.rados.rados.0.smithi161.stdout:74: writing smithi161112115-74 from 1622009 to 2348692 tid 2 2023-01-24T19:16:29.963 INFO:tasks.rados.rados.0.smithi161.stdout:74: writing smithi161112115-74 from 3121996 to 3226911 tid 3 2023-01-24T19:16:29.963 INFO:tasks.rados.rados.0.smithi161.stdout:75: write initial oid 75 2023-01-24T19:16:29.964 INFO:tasks.rados.rados.0.smithi161.stdout:75: seq_num 74 ranges {443718=615603,1564138=470341,2633066=610652} 2023-01-24T19:16:29.969 INFO:tasks.rados.rados.0.smithi161.stdout:75: writing smithi161112115-75 from 443718 to 1059321 tid 1 2023-01-24T19:16:29.974 INFO:tasks.rados.rados.0.smithi161.stdout:75: writing smithi161112115-75 from 1564138 to 2034479 tid 2 2023-01-24T19:16:29.981 INFO:tasks.rados.rados.0.smithi161.stdout:75: writing smithi161112115-75 from 2633066 to 3243718 tid 3 2023-01-24T19:16:29.981 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:29.982 INFO:tasks.rados.rados.0.smithi161.stdout:60: finishing write tid 1 to smithi161112115-60 2023-01-24T19:16:29.982 INFO:tasks.rados.rados.0.smithi161.stdout:60: finishing write tid 2 to smithi161112115-60 2023-01-24T19:16:29.982 INFO:tasks.rados.rados.0.smithi161.stdout:60: finishing write tid 3 to smithi161112115-60 2023-01-24T19:16:29.982 INFO:tasks.rados.rados.0.smithi161.stdout:60: finishing write tid 4 to smithi161112115-60 2023-01-24T19:16:29.982 INFO:tasks.rados.rados.0.smithi161.stdout:60: finishing write tid 5 to smithi161112115-60 2023-01-24T19:16:29.983 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 60 v 16 (ObjNum 59 snap 0 seq_num 59) dirty exists 2023-01-24T19:16:29.983 INFO:tasks.rados.rados.0.smithi161.stdout:60: left oid 60 (ObjNum 59 snap 0 seq_num 59) 2023-01-24T19:16:29.983 INFO:tasks.rados.rados.0.smithi161.stdout:61: finishing write tid 1 to smithi161112115-61 2023-01-24T19:16:29.983 INFO:tasks.rados.rados.0.smithi161.stdout:60: done (15 left) 2023-01-24T19:16:29.983 INFO:tasks.rados.rados.0.smithi161.stdout:76: write initial oid 76 2023-01-24T19:16:29.984 INFO:tasks.rados.rados.0.smithi161.stdout:76: seq_num 75 ranges {460525=690852,1622620=730352,3060489=200036} 2023-01-24T19:16:29.988 INFO:tasks.rados.rados.0.smithi161.stdout:76: writing smithi161112115-76 from 460525 to 1151377 tid 1 2023-01-24T19:16:29.995 INFO:tasks.rados.rados.0.smithi161.stdout:76: writing smithi161112115-76 from 1622620 to 2352972 tid 2 2023-01-24T19:16:30.000 INFO:tasks.rados.rados.0.smithi161.stdout:76: writing smithi161112115-76 from 3060489 to 3260525 tid 3 2023-01-24T19:16:30.000 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.000 INFO:tasks.rados.rados.0.smithi161.stdout:61: finishing write tid 2 to smithi161112115-61 2023-01-24T19:16:30.001 INFO:tasks.rados.rados.0.smithi161.stdout:61: finishing write tid 3 to smithi161112115-61 2023-01-24T19:16:30.001 INFO:tasks.rados.rados.0.smithi161.stdout:61: finishing write tid 4 to smithi161112115-61 2023-01-24T19:16:30.001 INFO:tasks.rados.rados.0.smithi161.stdout:61: finishing write tid 5 to smithi161112115-61 2023-01-24T19:16:30.001 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 61 v 24 (ObjNum 60 snap 0 seq_num 60) dirty exists 2023-01-24T19:16:30.001 INFO:tasks.rados.rados.0.smithi161.stdout:61: left oid 61 (ObjNum 60 snap 0 seq_num 60) 2023-01-24T19:16:30.002 INFO:tasks.rados.rados.0.smithi161.stdout:62: finishing write tid 1 to smithi161112115-62 2023-01-24T19:16:30.002 INFO:tasks.rados.rados.0.smithi161.stdout:61: done (15 left) 2023-01-24T19:16:30.002 INFO:tasks.rados.rados.0.smithi161.stdout:77: write initial oid 77 2023-01-24T19:16:30.002 INFO:tasks.rados.rados.0.smithi161.stdout:77: seq_num 76 ranges {477332=766101,1681102=474010,2687912=589420} 2023-01-24T19:16:30.007 INFO:tasks.rados.rados.0.smithi161.stdout:77: writing smithi161112115-77 from 477332 to 1243433 tid 1 2023-01-24T19:16:30.012 INFO:tasks.rados.rados.0.smithi161.stdout:77: writing smithi161112115-77 from 1681102 to 2155112 tid 2 2023-01-24T19:16:30.019 INFO:tasks.rados.rados.0.smithi161.stdout:77: writing smithi161112115-77 from 2687912 to 3277332 tid 3 2023-01-24T19:16:30.019 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.019 INFO:tasks.rados.rados.0.smithi161.stdout:62: finishing write tid 2 to smithi161112115-62 2023-01-24T19:16:30.019 INFO:tasks.rados.rados.0.smithi161.stdout:63: finishing write tid 1 to smithi161112115-63 2023-01-24T19:16:30.020 INFO:tasks.rados.rados.0.smithi161.stdout:62: finishing write tid 3 to smithi161112115-62 2023-01-24T19:16:30.020 INFO:tasks.rados.rados.0.smithi161.stdout:63: finishing write tid 2 to smithi161112115-63 2023-01-24T19:16:30.020 INFO:tasks.rados.rados.0.smithi161.stdout:63: finishing write tid 3 to smithi161112115-63 2023-01-24T19:16:30.020 INFO:tasks.rados.rados.0.smithi161.stdout:63: finishing write tid 4 to smithi161112115-63 2023-01-24T19:16:30.020 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 63 v 19 (ObjNum 62 snap 0 seq_num 62) dirty exists 2023-01-24T19:16:30.021 INFO:tasks.rados.rados.0.smithi161.stdout:63: left oid 63 (ObjNum 62 snap 0 seq_num 62) 2023-01-24T19:16:30.021 INFO:tasks.rados.rados.0.smithi161.stdout:62: finishing write tid 4 to smithi161112115-62 2023-01-24T19:16:30.021 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 62 v 43 (ObjNum 61 snap 0 seq_num 61) dirty exists 2023-01-24T19:16:30.021 INFO:tasks.rados.rados.0.smithi161.stdout:62: left oid 62 (ObjNum 61 snap 0 seq_num 61) 2023-01-24T19:16:30.021 INFO:tasks.rados.rados.0.smithi161.stdout:64: finishing write tid 1 to smithi161112115-64 2023-01-24T19:16:30.022 INFO:tasks.rados.rados.0.smithi161.stdout:62: done (15 left) 2023-01-24T19:16:30.022 INFO:tasks.rados.rados.0.smithi161.stdout:63: done (14 left) 2023-01-24T19:16:30.022 INFO:tasks.rados.rados.0.smithi161.stdout:78: write initial oid 78 2023-01-24T19:16:30.022 INFO:tasks.rados.rados.0.smithi161.stdout:78: seq_num 77 ranges {494139=557703,1455937=734021,2948041=346098} 2023-01-24T19:16:30.024 INFO:tasks.rados.rados.0.smithi161.stdout:78: writing smithi161112115-78 from 494139 to 1051842 tid 1 2023-01-24T19:16:30.031 INFO:tasks.rados.rados.0.smithi161.stdout:78: writing smithi161112115-78 from 1455937 to 2189958 tid 2 2023-01-24T19:16:30.037 INFO:tasks.rados.rados.0.smithi161.stdout:78: writing smithi161112115-78 from 2948041 to 3294139 tid 3 2023-01-24T19:16:30.037 INFO:tasks.rados.rados.0.smithi161.stdout:79: write initial oid 79 2023-01-24T19:16:30.037 INFO:tasks.rados.rados.0.smithi161.stdout:79: seq_num 78 ranges {510946=632952,1798066=477679,2742758=457217} 2023-01-24T19:16:30.043 INFO:tasks.rados.rados.0.smithi161.stdout:79: writing smithi161112115-79 from 510946 to 1143898 tid 1 2023-01-24T19:16:30.049 INFO:tasks.rados.rados.0.smithi161.stdout:79: writing smithi161112115-79 from 1798066 to 2275745 tid 2 2023-01-24T19:16:30.056 INFO:tasks.rados.rados.0.smithi161.stdout:79: writing smithi161112115-79 from 2742758 to 3199975 tid 3 2023-01-24T19:16:30.056 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.056 INFO:tasks.rados.rados.0.smithi161.stdout:64: finishing write tid 2 to smithi161112115-64 2023-01-24T19:16:30.057 INFO:tasks.rados.rados.0.smithi161.stdout:64: finishing write tid 3 to smithi161112115-64 2023-01-24T19:16:30.057 INFO:tasks.rados.rados.0.smithi161.stdout:64: finishing write tid 4 to smithi161112115-64 2023-01-24T19:16:30.057 INFO:tasks.rados.rados.0.smithi161.stdout:64: finishing write tid 5 to smithi161112115-64 2023-01-24T19:16:30.057 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 64 v 34 (ObjNum 63 snap 0 seq_num 63) dirty exists 2023-01-24T19:16:30.057 INFO:tasks.rados.rados.0.smithi161.stdout:64: left oid 64 (ObjNum 63 snap 0 seq_num 63) 2023-01-24T19:16:30.058 INFO:tasks.rados.rados.0.smithi161.stdout:65: finishing write tid 1 to smithi161112115-65 2023-01-24T19:16:30.058 INFO:tasks.rados.rados.0.smithi161.stdout:65: finishing write tid 2 to smithi161112115-65 2023-01-24T19:16:30.058 INFO:tasks.rados.rados.0.smithi161.stdout:65: finishing write tid 3 to smithi161112115-65 2023-01-24T19:16:30.058 INFO:tasks.rados.rados.0.smithi161.stdout:65: finishing write tid 4 to smithi161112115-65 2023-01-24T19:16:30.059 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 65 v 22 (ObjNum 64 snap 0 seq_num 64) dirty exists 2023-01-24T19:16:30.059 INFO:tasks.rados.rados.0.smithi161.stdout:65: left oid 65 (ObjNum 64 snap 0 seq_num 64) 2023-01-24T19:16:30.059 INFO:tasks.rados.rados.0.smithi161.stdout:66: finishing write tid 1 to smithi161112115-66 2023-01-24T19:16:30.059 INFO:tasks.rados.rados.0.smithi161.stdout:66: finishing write tid 2 to smithi161112115-66 2023-01-24T19:16:30.059 INFO:tasks.rados.rados.0.smithi161.stdout:66: finishing write tid 3 to smithi161112115-66 2023-01-24T19:16:30.060 INFO:tasks.rados.rados.0.smithi161.stdout:66: finishing write tid 4 to smithi161112115-66 2023-01-24T19:16:30.060 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 66 v 36 (ObjNum 65 snap 0 seq_num 65) dirty exists 2023-01-24T19:16:30.060 INFO:tasks.rados.rados.0.smithi161.stdout:66: left oid 66 (ObjNum 65 snap 0 seq_num 65) 2023-01-24T19:16:30.060 INFO:tasks.rados.rados.0.smithi161.stdout:67: finishing write tid 1 to smithi161112115-67 2023-01-24T19:16:30.060 INFO:tasks.rados.rados.0.smithi161.stdout:67: finishing write tid 2 to smithi161112115-67 2023-01-24T19:16:30.060 INFO:tasks.rados.rados.0.smithi161.stdout:67: finishing write tid 3 to smithi161112115-67 2023-01-24T19:16:30.061 INFO:tasks.rados.rados.0.smithi161.stdout:67: finishing write tid 4 to smithi161112115-67 2023-01-24T19:16:30.061 INFO:tasks.rados.rados.0.smithi161.stdout:67: finishing write tid 5 to smithi161112115-67 2023-01-24T19:16:30.061 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 67 v 40 (ObjNum 66 snap 0 seq_num 66) dirty exists 2023-01-24T19:16:30.061 INFO:tasks.rados.rados.0.smithi161.stdout:67: left oid 67 (ObjNum 66 snap 0 seq_num 66) 2023-01-24T19:16:30.061 INFO:tasks.rados.rados.0.smithi161.stdout:68: finishing write tid 1 to smithi161112115-68 2023-01-24T19:16:30.062 INFO:tasks.rados.rados.0.smithi161.stdout:68: finishing write tid 2 to smithi161112115-68 2023-01-24T19:16:30.062 INFO:tasks.rados.rados.0.smithi161.stdout:68: finishing write tid 3 to smithi161112115-68 2023-01-24T19:16:30.062 INFO:tasks.rados.rados.0.smithi161.stdout:68: finishing write tid 4 to smithi161112115-68 2023-01-24T19:16:30.062 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 68 v 43 (ObjNum 67 snap 0 seq_num 67) dirty exists 2023-01-24T19:16:30.062 INFO:tasks.rados.rados.0.smithi161.stdout:68: left oid 68 (ObjNum 67 snap 0 seq_num 67) 2023-01-24T19:16:30.063 INFO:tasks.rados.rados.0.smithi161.stdout:69: finishing write tid 1 to smithi161112115-69 2023-01-24T19:16:30.063 INFO:tasks.rados.rados.0.smithi161.stdout:69: finishing write tid 2 to smithi161112115-69 2023-01-24T19:16:30.063 INFO:tasks.rados.rados.0.smithi161.stdout:69: finishing write tid 3 to smithi161112115-69 2023-01-24T19:16:30.063 INFO:tasks.rados.rados.0.smithi161.stdout:70: finishing write tid 1 to smithi161112115-70 2023-01-24T19:16:30.063 INFO:tasks.rados.rados.0.smithi161.stdout:69: finishing write tid 4 to smithi161112115-69 2023-01-24T19:16:30.064 INFO:tasks.rados.rados.0.smithi161.stdout:69: finishing write tid 5 to smithi161112115-69 2023-01-24T19:16:30.064 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 69 v 29 (ObjNum 68 snap 0 seq_num 68) dirty exists 2023-01-24T19:16:30.064 INFO:tasks.rados.rados.0.smithi161.stdout:69: left oid 69 (ObjNum 68 snap 0 seq_num 68) 2023-01-24T19:16:30.064 INFO:tasks.rados.rados.0.smithi161.stdout:70: finishing write tid 2 to smithi161112115-70 2023-01-24T19:16:30.064 INFO:tasks.rados.rados.0.smithi161.stdout:70: finishing write tid 3 to smithi161112115-70 2023-01-24T19:16:30.065 INFO:tasks.rados.rados.0.smithi161.stdout:70: finishing write tid 4 to smithi161112115-70 2023-01-24T19:16:30.065 INFO:tasks.rados.rados.0.smithi161.stdout:70: finishing write tid 5 to smithi161112115-70 2023-01-24T19:16:30.065 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 70 v 47 (ObjNum 69 snap 0 seq_num 69) dirty exists 2023-01-24T19:16:30.065 INFO:tasks.rados.rados.0.smithi161.stdout:70: left oid 70 (ObjNum 69 snap 0 seq_num 69) 2023-01-24T19:16:30.065 INFO:tasks.rados.rados.0.smithi161.stdout:71: finishing write tid 1 to smithi161112115-71 2023-01-24T19:16:30.066 INFO:tasks.rados.rados.0.smithi161.stdout:64: done (15 left) 2023-01-24T19:16:30.066 INFO:tasks.rados.rados.0.smithi161.stdout:65: done (14 left) 2023-01-24T19:16:30.066 INFO:tasks.rados.rados.0.smithi161.stdout:66: done (13 left) 2023-01-24T19:16:30.066 INFO:tasks.rados.rados.0.smithi161.stdout:67: done (12 left) 2023-01-24T19:16:30.066 INFO:tasks.rados.rados.0.smithi161.stdout:68: done (11 left) 2023-01-24T19:16:30.067 INFO:tasks.rados.rados.0.smithi161.stdout:69: done (10 left) 2023-01-24T19:16:30.067 INFO:tasks.rados.rados.0.smithi161.stdout:70: done (9 left) 2023-01-24T19:16:30.067 INFO:tasks.rados.rados.0.smithi161.stdout:80: write initial oid 80 2023-01-24T19:16:30.067 INFO:tasks.rados.rados.0.smithi161.stdout:80: seq_num 79 ranges {527753=708201,1856548=737690,3286534=41219} 2023-01-24T19:16:30.068 INFO:tasks.rados.rados.0.smithi161.stdout:80: writing smithi161112115-80 from 527753 to 1235954 tid 1 2023-01-24T19:16:30.070 INFO:tasks.rados.rados.0.smithi161.stdout:80: writing smithi161112115-80 from 1856548 to 2594238 tid 2 2023-01-24T19:16:30.074 INFO:tasks.rados.rados.0.smithi161.stdout:80: writing smithi161112115-80 from 3286534 to 3327753 tid 3 2023-01-24T19:16:30.074 INFO:tasks.rados.rados.0.smithi161.stdout:81: write initial oid 81 2023-01-24T19:16:30.074 INFO:tasks.rados.rados.0.smithi161.stdout:81: seq_num 80 ranges {544560=783450,1915030=481348,2797604=479761} 2023-01-24T19:16:30.081 INFO:tasks.rados.rados.0.smithi161.stdout:81: writing smithi161112115-81 from 544560 to 1328010 tid 1 2023-01-24T19:16:30.087 INFO:tasks.rados.rados.0.smithi161.stdout:81: writing smithi161112115-81 from 1915030 to 2396378 tid 2 2023-01-24T19:16:30.092 INFO:tasks.rados.rados.0.smithi161.stdout:81: writing smithi161112115-81 from 2797604 to 3277365 tid 3 2023-01-24T19:16:30.093 INFO:tasks.rados.rados.0.smithi161.stdout:82: write initial oid 82 2023-01-24T19:16:30.093 INFO:tasks.rados.rados.0.smithi161.stdout:82: seq_num 81 ranges {561367=458699,1573512=741359,2941380=419987} 2023-01-24T19:16:30.098 INFO:tasks.rados.rados.0.smithi161.stdout:82: writing smithi161112115-82 from 561367 to 1020066 tid 1 2023-01-24T19:16:30.105 INFO:tasks.rados.rados.0.smithi161.stdout:82: writing smithi161112115-82 from 1573512 to 2314871 tid 2 2023-01-24T19:16:30.112 INFO:tasks.rados.rados.0.smithi161.stdout:82: writing smithi161112115-82 from 2941380 to 3361367 tid 3 2023-01-24T19:16:30.112 INFO:tasks.rados.rados.0.smithi161.stdout:83: write initial oid 83 2023-01-24T19:16:30.112 INFO:tasks.rados.rados.0.smithi161.stdout:83: seq_num 82 ranges {578174=533948,1515641=485017,2736097=502305} 2023-01-24T19:16:30.118 INFO:tasks.rados.rados.0.smithi161.stdout:83: writing smithi161112115-83 from 578174 to 1112122 tid 1 2023-01-24T19:16:30.123 INFO:tasks.rados.rados.0.smithi161.stdout:83: writing smithi161112115-83 from 1515641 to 2000658 tid 2 2023-01-24T19:16:30.130 INFO:tasks.rados.rados.0.smithi161.stdout:83: writing smithi161112115-83 from 2736097 to 3238402 tid 3 2023-01-24T19:16:30.130 INFO:tasks.rados.rados.0.smithi161.stdout:84: write initial oid 84 2023-01-24T19:16:30.130 INFO:tasks.rados.rados.0.smithi161.stdout:84: seq_num 83 ranges {594981=609197,1974123=745028,3279873=115108} 2023-01-24T19:16:30.136 INFO:tasks.rados.rados.0.smithi161.stdout:84: writing smithi161112115-84 from 594981 to 1204178 tid 1 2023-01-24T19:16:30.145 INFO:tasks.rados.rados.0.smithi161.stdout:84: writing smithi161112115-84 from 1974123 to 2719151 tid 2 2023-01-24T19:16:30.148 INFO:tasks.rados.rados.0.smithi161.stdout:84: writing smithi161112115-84 from 3279873 to 3394981 tid 3 2023-01-24T19:16:30.149 INFO:tasks.rados.rados.0.smithi161.stdout:85: write initial oid 85 2023-01-24T19:16:30.149 INFO:tasks.rados.rados.0.smithi161.stdout:85: seq_num 84 ranges {611788=400799,1748958=488686,2907296=504492} 2023-01-24T19:16:30.154 INFO:tasks.rados.rados.0.smithi161.stdout:85: writing smithi161112115-85 from 611788 to 1012587 tid 1 2023-01-24T19:16:30.161 INFO:tasks.rados.rados.0.smithi161.stdout:85: writing smithi161112115-85 from 1748958 to 2237644 tid 2 2023-01-24T19:16:30.168 INFO:tasks.rados.rados.0.smithi161.stdout:85: writing smithi161112115-85 from 2907296 to 3411788 tid 3 2023-01-24T19:16:30.168 INFO:tasks.rados.rados.0.smithi161.stdout:86: write initial oid 86 2023-01-24T19:16:30.168 INFO:tasks.rados.rados.0.smithi161.stdout:86: seq_num 85 ranges {628595=476048,1807440=632344,2934719=452474} 2023-01-24T19:16:30.173 INFO:tasks.rados.rados.0.smithi161.stdout:86: writing smithi161112115-86 from 628595 to 1104643 tid 1 2023-01-24T19:16:30.181 INFO:tasks.rados.rados.0.smithi161.stdout:86: writing smithi161112115-86 from 1807440 to 2439784 tid 2 2023-01-24T19:16:30.187 INFO:tasks.rados.rados.0.smithi161.stdout:86: writing smithi161112115-86 from 2934719 to 3387193 tid 3 2023-01-24T19:16:30.187 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.188 INFO:tasks.rados.rados.0.smithi161.stdout:71: finishing write tid 2 to smithi161112115-71 2023-01-24T19:16:30.188 INFO:tasks.rados.rados.0.smithi161.stdout:71: finishing write tid 3 to smithi161112115-71 2023-01-24T19:16:30.188 INFO:tasks.rados.rados.0.smithi161.stdout:71: finishing write tid 4 to smithi161112115-71 2023-01-24T19:16:30.188 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 71 v 50 (ObjNum 70 snap 0 seq_num 70) dirty exists 2023-01-24T19:16:30.188 INFO:tasks.rados.rados.0.smithi161.stdout:71: left oid 71 (ObjNum 70 snap 0 seq_num 70) 2023-01-24T19:16:30.189 INFO:tasks.rados.rados.0.smithi161.stdout:72: finishing write tid 1 to smithi161112115-72 2023-01-24T19:16:30.189 INFO:tasks.rados.rados.0.smithi161.stdout:72: finishing write tid 2 to smithi161112115-72 2023-01-24T19:16:30.189 INFO:tasks.rados.rados.0.smithi161.stdout:72: finishing write tid 3 to smithi161112115-72 2023-01-24T19:16:30.189 INFO:tasks.rados.rados.0.smithi161.stdout:72: finishing write tid 4 to smithi161112115-72 2023-01-24T19:16:30.189 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 72 v 53 (ObjNum 71 snap 0 seq_num 71) dirty exists 2023-01-24T19:16:30.190 INFO:tasks.rados.rados.0.smithi161.stdout:72: left oid 72 (ObjNum 71 snap 0 seq_num 71) 2023-01-24T19:16:30.190 INFO:tasks.rados.rados.0.smithi161.stdout:73: finishing write tid 1 to smithi161112115-73 2023-01-24T19:16:30.190 INFO:tasks.rados.rados.0.smithi161.stdout:73: finishing write tid 2 to smithi161112115-73 2023-01-24T19:16:30.190 INFO:tasks.rados.rados.0.smithi161.stdout:73: finishing write tid 3 to smithi161112115-73 2023-01-24T19:16:30.190 INFO:tasks.rados.rados.0.smithi161.stdout:73: finishing write tid 4 to smithi161112115-73 2023-01-24T19:16:30.190 INFO:tasks.rados.rados.0.smithi161.stdout:73: finishing write tid 5 to smithi161112115-73 2023-01-24T19:16:30.191 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 73 v 33 (ObjNum 72 snap 0 seq_num 72) dirty exists 2023-01-24T19:16:30.191 INFO:tasks.rados.rados.0.smithi161.stdout:73: left oid 73 (ObjNum 72 snap 0 seq_num 72) 2023-01-24T19:16:30.191 INFO:tasks.rados.rados.0.smithi161.stdout:71: done (15 left) 2023-01-24T19:16:30.191 INFO:tasks.rados.rados.0.smithi161.stdout:72: done (14 left) 2023-01-24T19:16:30.191 INFO:tasks.rados.rados.0.smithi161.stdout:73: done (13 left) 2023-01-24T19:16:30.192 INFO:tasks.rados.rados.0.smithi161.stdout:87: write initial oid 87 2023-01-24T19:16:30.192 INFO:tasks.rados.rados.0.smithi161.stdout:87: seq_num 86 ranges {645402=551297,1749569=492355,2845789=599613} 2023-01-24T19:16:30.193 INFO:tasks.rados.rados.0.smithi161.stdout:87: writing smithi161112115-87 from 645402 to 1196699 tid 1 2023-01-24T19:16:30.199 INFO:tasks.rados.rados.0.smithi161.stdout:87: writing smithi161112115-87 from 1749569 to 2241924 tid 2 2023-01-24T19:16:30.206 INFO:tasks.rados.rados.0.smithi161.stdout:87: writing smithi161112115-87 from 2845789 to 3445402 tid 3 2023-01-24T19:16:30.206 INFO:tasks.rados.rados.0.smithi161.stdout:88: write initial oid 88 2023-01-24T19:16:30.206 INFO:tasks.rados.rados.0.smithi161.stdout:88: seq_num 87 ranges {662209=626546,1808051=636013,2873212=588997} 2023-01-24T19:16:30.213 INFO:tasks.rados.rados.0.smithi161.stdout:88: writing smithi161112115-88 from 662209 to 1288755 tid 1 2023-01-24T19:16:30.220 INFO:tasks.rados.rados.0.smithi161.stdout:88: writing smithi161112115-88 from 1808051 to 2444064 tid 2 2023-01-24T19:16:30.225 INFO:tasks.rados.rados.0.smithi161.stdout:88: writing smithi161112115-88 from 2873212 to 3462209 tid 3 2023-01-24T19:16:30.225 INFO:tasks.rados.rados.0.smithi161.stdout:89: write initial oid 89 2023-01-24T19:16:30.226 INFO:tasks.rados.rados.0.smithi161.stdout:89: seq_num 88 ranges {679016=701795,1866533=496024,2900635=402643} 2023-01-24T19:16:30.233 INFO:tasks.rados.rados.0.smithi161.stdout:89: writing smithi161112115-89 from 679016 to 1380811 tid 1 2023-01-24T19:16:30.238 INFO:tasks.rados.rados.0.smithi161.stdout:89: writing smithi161112115-89 from 1866533 to 2362557 tid 2 2023-01-24T19:16:30.244 INFO:tasks.rados.rados.0.smithi161.stdout:89: writing smithi161112115-89 from 2900635 to 3303278 tid 3 2023-01-24T19:16:30.244 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.244 INFO:tasks.rados.rados.0.smithi161.stdout:74: finishing write tid 1 to smithi161112115-74 2023-01-24T19:16:30.244 INFO:tasks.rados.rados.0.smithi161.stdout:74: finishing write tid 2 to smithi161112115-74 2023-01-24T19:16:30.245 INFO:tasks.rados.rados.0.smithi161.stdout:74: finishing write tid 3 to smithi161112115-74 2023-01-24T19:16:30.245 INFO:tasks.rados.rados.0.smithi161.stdout:74: finishing write tid 4 to smithi161112115-74 2023-01-24T19:16:30.245 INFO:tasks.rados.rados.0.smithi161.stdout:74: finishing write tid 5 to smithi161112115-74 2023-01-24T19:16:30.245 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 74 v 38 (ObjNum 73 snap 0 seq_num 73) dirty exists 2023-01-24T19:16:30.245 INFO:tasks.rados.rados.0.smithi161.stdout:74: left oid 74 (ObjNum 73 snap 0 seq_num 73) 2023-01-24T19:16:30.246 INFO:tasks.rados.rados.0.smithi161.stdout:75: finishing write tid 1 to smithi161112115-75 2023-01-24T19:16:30.246 INFO:tasks.rados.rados.0.smithi161.stdout:75: finishing write tid 2 to smithi161112115-75 2023-01-24T19:16:30.246 INFO:tasks.rados.rados.0.smithi161.stdout:74: done (15 left) 2023-01-24T19:16:30.246 INFO:tasks.rados.rados.0.smithi161.stdout:90: write initial oid 90 2023-01-24T19:16:30.246 INFO:tasks.rados.rados.0.smithi161.stdout:90: seq_num 89 ranges {695823=777044,1925015=639682,3328058=167765} 2023-01-24T19:16:30.251 INFO:tasks.rados.rados.0.smithi161.stdout:90: writing smithi161112115-90 from 695823 to 1472867 tid 1 2023-01-24T19:16:30.258 INFO:tasks.rados.rados.0.smithi161.stdout:90: writing smithi161112115-90 from 1925015 to 2564697 tid 2 2023-01-24T19:16:30.263 INFO:tasks.rados.rados.0.smithi161.stdout:90: writing smithi161112115-90 from 3328058 to 3495823 tid 3 2023-01-24T19:16:30.264 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.264 INFO:tasks.rados.rados.0.smithi161.stdout:75: finishing write tid 3 to smithi161112115-75 2023-01-24T19:16:30.264 INFO:tasks.rados.rados.0.smithi161.stdout:75: finishing write tid 4 to smithi161112115-75 2023-01-24T19:16:30.264 INFO:tasks.rados.rados.0.smithi161.stdout:75: finishing write tid 5 to smithi161112115-75 2023-01-24T19:16:30.264 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 75 v 57 (ObjNum 74 snap 0 seq_num 74) dirty exists 2023-01-24T19:16:30.265 INFO:tasks.rados.rados.0.smithi161.stdout:75: left oid 75 (ObjNum 74 snap 0 seq_num 74) 2023-01-24T19:16:30.265 INFO:tasks.rados.rados.0.smithi161.stdout:76: finishing write tid 1 to smithi161112115-76 2023-01-24T19:16:30.265 INFO:tasks.rados.rados.0.smithi161.stdout:76: finishing write tid 2 to smithi161112115-76 2023-01-24T19:16:30.265 INFO:tasks.rados.rados.0.smithi161.stdout:76: finishing write tid 3 to smithi161112115-76 2023-01-24T19:16:30.265 INFO:tasks.rados.rados.0.smithi161.stdout:76: finishing write tid 4 to smithi161112115-76 2023-01-24T19:16:30.266 INFO:tasks.rados.rados.0.smithi161.stdout:75: done (15 left) 2023-01-24T19:16:30.266 INFO:tasks.rados.rados.0.smithi161.stdout:91: write initial oid 91 2023-01-24T19:16:30.266 INFO:tasks.rados.rados.0.smithi161.stdout:91: seq_num 90 ranges {712630=452293,1583497=499693,2555481=425187} 2023-01-24T19:16:30.269 INFO:tasks.rados.rados.0.smithi161.stdout:91: writing smithi161112115-91 from 712630 to 1164923 tid 1 2023-01-24T19:16:30.275 INFO:tasks.rados.rados.0.smithi161.stdout:91: writing smithi161112115-91 from 1583497 to 2083190 tid 2 2023-01-24T19:16:30.280 INFO:tasks.rados.rados.0.smithi161.stdout:91: writing smithi161112115-91 from 2555481 to 2980668 tid 3 2023-01-24T19:16:30.280 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.280 INFO:tasks.rados.rados.0.smithi161.stdout:76: finishing write tid 5 to smithi161112115-76 2023-01-24T19:16:30.281 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 76 v 29 (ObjNum 75 snap 0 seq_num 75) dirty exists 2023-01-24T19:16:30.281 INFO:tasks.rados.rados.0.smithi161.stdout:76: left oid 76 (ObjNum 75 snap 0 seq_num 75) 2023-01-24T19:16:30.281 INFO:tasks.rados.rados.0.smithi161.stdout:77: finishing write tid 1 to smithi161112115-77 2023-01-24T19:16:30.281 INFO:tasks.rados.rados.0.smithi161.stdout:76: done (15 left) 2023-01-24T19:16:30.281 INFO:tasks.rados.rados.0.smithi161.stdout:92: write initial oid 92 2023-01-24T19:16:30.282 INFO:tasks.rados.rados.0.smithi161.stdout:92: seq_num 91 ranges {729437=527542,1925626=643351,3266551=262886} 2023-01-24T19:16:30.287 INFO:tasks.rados.rados.0.smithi161.stdout:92: writing smithi161112115-92 from 729437 to 1256979 tid 1 2023-01-24T19:16:30.294 INFO:tasks.rados.rados.0.smithi161.stdout:92: writing smithi161112115-92 from 1925626 to 2568977 tid 2 2023-01-24T19:16:30.301 INFO:tasks.rados.rados.0.smithi161.stdout:92: writing smithi161112115-92 from 3266551 to 3529437 tid 3 2023-01-24T19:16:30.301 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.301 INFO:tasks.rados.rados.0.smithi161.stdout:77: finishing write tid 2 to smithi161112115-77 2023-01-24T19:16:30.301 INFO:tasks.rados.rados.0.smithi161.stdout:77: finishing write tid 3 to smithi161112115-77 2023-01-24T19:16:30.302 INFO:tasks.rados.rados.0.smithi161.stdout:77: finishing write tid 4 to smithi161112115-77 2023-01-24T19:16:30.302 INFO:tasks.rados.rados.0.smithi161.stdout:77: finishing write tid 5 to smithi161112115-77 2023-01-24T19:16:30.302 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 77 v 37 (ObjNum 76 snap 0 seq_num 76) dirty exists 2023-01-24T19:16:30.302 INFO:tasks.rados.rados.0.smithi161.stdout:77: left oid 77 (ObjNum 76 snap 0 seq_num 76) 2023-01-24T19:16:30.302 INFO:tasks.rados.rados.0.smithi161.stdout:78: finishing write tid 1 to smithi161112115-78 2023-01-24T19:16:30.303 INFO:tasks.rados.rados.0.smithi161.stdout:78: finishing write tid 2 to smithi161112115-78 2023-01-24T19:16:30.303 INFO:tasks.rados.rados.0.smithi161.stdout:78: finishing write tid 3 to smithi161112115-78 2023-01-24T19:16:30.303 INFO:tasks.rados.rados.0.smithi161.stdout:78: finishing write tid 4 to smithi161112115-78 2023-01-24T19:16:30.303 INFO:tasks.rados.rados.0.smithi161.stdout:78: finishing write tid 5 to smithi161112115-78 2023-01-24T19:16:30.303 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 78 v 61 (ObjNum 77 snap 0 seq_num 77) dirty exists 2023-01-24T19:16:30.304 INFO:tasks.rados.rados.0.smithi161.stdout:78: left oid 78 (ObjNum 77 snap 0 seq_num 77) 2023-01-24T19:16:30.304 INFO:tasks.rados.rados.0.smithi161.stdout:79: finishing write tid 1 to smithi161112115-79 2023-01-24T19:16:30.304 INFO:tasks.rados.rados.0.smithi161.stdout:79: finishing write tid 2 to smithi161112115-79 2023-01-24T19:16:30.304 INFO:tasks.rados.rados.0.smithi161.stdout:79: finishing write tid 3 to smithi161112115-79 2023-01-24T19:16:30.304 INFO:tasks.rados.rados.0.smithi161.stdout:79: finishing write tid 4 to smithi161112115-79 2023-01-24T19:16:30.305 INFO:tasks.rados.rados.0.smithi161.stdout:79: finishing write tid 5 to smithi161112115-79 2023-01-24T19:16:30.305 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 79 v 65 (ObjNum 78 snap 0 seq_num 78) dirty exists 2023-01-24T19:16:30.305 INFO:tasks.rados.rados.0.smithi161.stdout:79: left oid 79 (ObjNum 78 snap 0 seq_num 78) 2023-01-24T19:16:30.305 INFO:tasks.rados.rados.0.smithi161.stdout:80: finishing write tid 1 to smithi161112115-80 2023-01-24T19:16:30.305 INFO:tasks.rados.rados.0.smithi161.stdout:80: finishing write tid 2 to smithi161112115-80 2023-01-24T19:16:30.306 INFO:tasks.rados.rados.0.smithi161.stdout:77: done (15 left) 2023-01-24T19:16:30.306 INFO:tasks.rados.rados.0.smithi161.stdout:78: done (14 left) 2023-01-24T19:16:30.306 INFO:tasks.rados.rados.0.smithi161.stdout:79: done (13 left) 2023-01-24T19:16:30.306 INFO:tasks.rados.rados.0.smithi161.stdout:93: write initial oid 93 2023-01-24T19:16:30.306 INFO:tasks.rados.rados.0.smithi161.stdout:93: seq_num 92 ranges {746244=719144,2100461=503362,3126680=419564} 2023-01-24T19:16:30.307 INFO:tasks.rados.rados.0.smithi161.stdout:93: writing smithi161112115-93 from 746244 to 1465388 tid 1 2023-01-24T19:16:30.314 INFO:tasks.rados.rados.0.smithi161.stdout:93: writing smithi161112115-93 from 2100461 to 2603823 tid 2 2023-01-24T19:16:30.319 INFO:tasks.rados.rados.0.smithi161.stdout:93: writing smithi161112115-93 from 3126680 to 3546244 tid 3 2023-01-24T19:16:30.320 INFO:tasks.rados.rados.0.smithi161.stdout:94: write initial oid 94 2023-01-24T19:16:30.320 INFO:tasks.rados.rados.0.smithi161.stdout:80: finishing write tid 3 to smithi161112115-80 2023-01-24T19:16:30.320 INFO:tasks.rados.rados.0.smithi161.stdout:80: finishing write tid 4 to smithi161112115-80 2023-01-24T19:16:30.320 INFO:tasks.rados.rados.0.smithi161.stdout:80: finishing write tid 5 to smithi161112115-80 2023-01-24T19:16:30.320 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 80 v 69 (ObjNum 79 snap 0 seq_num 79) dirty exists 2023-01-24T19:16:30.321 INFO:tasks.rados.rados.0.smithi161.stdout:80: left oid 80 (ObjNum 79 snap 0 seq_num 79) 2023-01-24T19:16:30.321 INFO:tasks.rados.rados.0.smithi161.stdout:81: finishing write tid 1 to smithi161112115-81 2023-01-24T19:16:30.321 INFO:tasks.rados.rados.0.smithi161.stdout:81: finishing write tid 2 to smithi161112115-81 2023-01-24T19:16:30.321 INFO:tasks.rados.rados.0.smithi161.stdout:81: finishing write tid 3 to smithi161112115-81 2023-01-24T19:16:30.322 INFO:tasks.rados.rados.0.smithi161.stdout:81: finishing write tid 4 to smithi161112115-81 2023-01-24T19:16:30.322 INFO:tasks.rados.rados.0.smithi161.stdout:81: finishing write tid 5 to smithi161112115-81 2023-01-24T19:16:30.322 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 81 v 33 (ObjNum 80 snap 0 seq_num 80) dirty exists 2023-01-24T19:16:30.322 INFO:tasks.rados.rados.0.smithi161.stdout:81: left oid 81 (ObjNum 80 snap 0 seq_num 80) 2023-01-24T19:16:30.322 INFO:tasks.rados.rados.0.smithi161.stdout:82: finishing write tid 1 to smithi161112115-82 2023-01-24T19:16:30.323 INFO:tasks.rados.rados.0.smithi161.stdout:82: finishing write tid 2 to smithi161112115-82 2023-01-24T19:16:30.323 INFO:tasks.rados.rados.0.smithi161.stdout:82: finishing write tid 3 to smithi161112115-82 2023-01-24T19:16:30.323 INFO:tasks.rados.rados.0.smithi161.stdout:82: finishing write tid 4 to smithi161112115-82 2023-01-24T19:16:30.323 INFO:tasks.rados.rados.0.smithi161.stdout:82: finishing write tid 5 to smithi161112115-82 2023-01-24T19:16:30.323 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 82 v 31 (ObjNum 81 snap 0 seq_num 81) dirty exists 2023-01-24T19:16:30.324 INFO:tasks.rados.rados.0.smithi161.stdout:82: left oid 82 (ObjNum 81 snap 0 seq_num 81) 2023-01-24T19:16:30.324 INFO:tasks.rados.rados.0.smithi161.stdout:83: finishing write tid 1 to smithi161112115-83 2023-01-24T19:16:30.324 INFO:tasks.rados.rados.0.smithi161.stdout:94: seq_num 93 ranges {763051=794393,2158943=647020,3437750=125301} 2023-01-24T19:16:30.327 INFO:tasks.rados.rados.0.smithi161.stdout:94: writing smithi161112115-94 from 763051 to 1557444 tid 1 2023-01-24T19:16:30.334 INFO:tasks.rados.rados.0.smithi161.stdout:94: writing smithi161112115-94 from 2158943 to 2805963 tid 2 2023-01-24T19:16:30.339 INFO:tasks.rados.rados.0.smithi161.stdout:94: writing smithi161112115-94 from 3437750 to 3563051 tid 3 2023-01-24T19:16:30.339 INFO:tasks.rados.rados.0.smithi161.stdout:80: done (14 left) 2023-01-24T19:16:30.339 INFO:tasks.rados.rados.0.smithi161.stdout:81: done (13 left) 2023-01-24T19:16:30.339 INFO:tasks.rados.rados.0.smithi161.stdout:82: done (12 left) 2023-01-24T19:16:30.339 INFO:tasks.rados.rados.0.smithi161.stdout:95: write initial oid 95 2023-01-24T19:16:30.340 INFO:tasks.rados.rados.0.smithi161.stdout:95: seq_num 94 ranges {779858=469642,1817425=507031,2781526=586628} 2023-01-24T19:16:30.345 INFO:tasks.rados.rados.0.smithi161.stdout:95: writing smithi161112115-95 from 779858 to 1249500 tid 1 2023-01-24T19:16:30.351 INFO:tasks.rados.rados.0.smithi161.stdout:95: writing smithi161112115-95 from 1817425 to 2324456 tid 2 2023-01-24T19:16:30.357 INFO:tasks.rados.rados.0.smithi161.stdout:95: writing smithi161112115-95 from 2781526 to 3368154 tid 3 2023-01-24T19:16:30.357 INFO:tasks.rados.rados.0.smithi161.stdout:96: write initial oid 96 2023-01-24T19:16:30.358 INFO:tasks.rados.rados.0.smithi161.stdout:96: seq_num 95 ranges {796665=544891,1759554=650689,2976243=620422} 2023-01-24T19:16:30.364 INFO:tasks.rados.rados.0.smithi161.stdout:96: writing smithi161112115-96 from 796665 to 1341556 tid 1 2023-01-24T19:16:30.370 INFO:tasks.rados.rados.0.smithi161.stdout:96: writing smithi161112115-96 from 1759554 to 2410243 tid 2 2023-01-24T19:16:30.377 INFO:tasks.rados.rados.0.smithi161.stdout:96: writing smithi161112115-96 from 2976243 to 3596665 tid 3 2023-01-24T19:16:30.378 INFO:tasks.rados.rados.0.smithi161.stdout:97: write initial oid 97 2023-01-24T19:16:30.378 INFO:tasks.rados.rados.0.smithi161.stdout:83: finishing write tid 2 to smithi161112115-83 2023-01-24T19:16:30.378 INFO:tasks.rados.rados.0.smithi161.stdout:83: finishing write tid 3 to smithi161112115-83 2023-01-24T19:16:30.378 INFO:tasks.rados.rados.0.smithi161.stdout:97: seq_num 96 ranges {413472=620140,1818036=510700,3120019=493453} 2023-01-24T19:16:30.383 INFO:tasks.rados.rados.0.smithi161.stdout:97: writing smithi161112115-97 from 413472 to 1033612 tid 1 2023-01-24T19:16:30.390 INFO:tasks.rados.rados.0.smithi161.stdout:97: writing smithi161112115-97 from 1818036 to 2328736 tid 2 2023-01-24T19:16:30.398 INFO:tasks.rados.rados.0.smithi161.stdout:97: writing smithi161112115-97 from 3120019 to 3613472 tid 3 2023-01-24T19:16:30.398 INFO:tasks.rados.rados.0.smithi161.stdout:98: write initial oid 98 2023-01-24T19:16:30.398 INFO:tasks.rados.rados.0.smithi161.stdout:83: finishing write tid 4 to smithi161112115-83 2023-01-24T19:16:30.398 INFO:tasks.rados.rados.0.smithi161.stdout:83: finishing write tid 5 to smithi161112115-83 2023-01-24T19:16:30.399 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 83 v 73 (ObjNum 82 snap 0 seq_num 82) dirty exists 2023-01-24T19:16:30.399 INFO:tasks.rados.rados.0.smithi161.stdout:83: left oid 83 (ObjNum 82 snap 0 seq_num 82) 2023-01-24T19:16:30.399 INFO:tasks.rados.rados.0.smithi161.stdout:84: finishing write tid 1 to smithi161112115-84 2023-01-24T19:16:30.399 INFO:tasks.rados.rados.0.smithi161.stdout:84: finishing write tid 2 to smithi161112115-84 2023-01-24T19:16:30.399 INFO:tasks.rados.rados.0.smithi161.stdout:98: seq_num 97 ranges {430279=695389,1876518=654358,3031089=536797} 2023-01-24T19:16:30.404 INFO:tasks.rados.rados.0.smithi161.stdout:98: writing smithi161112115-98 from 430279 to 1125668 tid 1 2023-01-24T19:16:30.412 INFO:tasks.rados.rados.0.smithi161.stdout:98: writing smithi161112115-98 from 1876518 to 2530876 tid 2 2023-01-24T19:16:30.418 INFO:tasks.rados.rados.0.smithi161.stdout:98: writing smithi161112115-98 from 3031089 to 3567886 tid 3 2023-01-24T19:16:30.418 INFO:tasks.rados.rados.0.smithi161.stdout:83: done (15 left) 2023-01-24T19:16:30.418 INFO:tasks.rados.rados.0.smithi161.stdout:99: write initial oid 99 2023-01-24T19:16:30.419 INFO:tasks.rados.rados.0.smithi161.stdout:99: seq_num 98 ranges {447086=770638,1935000=798016,3458512=188574} 2023-01-24T19:16:30.424 INFO:tasks.rados.rados.0.smithi161.stdout:99: writing smithi161112115-99 from 447086 to 1217724 tid 1 2023-01-24T19:16:30.433 INFO:tasks.rados.rados.0.smithi161.stdout:99: writing smithi161112115-99 from 1935000 to 2733016 tid 2 2023-01-24T19:16:30.438 INFO:tasks.rados.rados.0.smithi161.stdout:99: writing smithi161112115-99 from 3458512 to 3647086 tid 3 2023-01-24T19:16:30.438 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.439 INFO:tasks.rados.rados.0.smithi161.stdout:85: finishing write tid 1 to smithi161112115-85 2023-01-24T19:16:30.439 INFO:tasks.rados.rados.0.smithi161.stdout:84: finishing write tid 3 to smithi161112115-84 2023-01-24T19:16:30.439 INFO:tasks.rados.rados.0.smithi161.stdout:84: finishing write tid 4 to smithi161112115-84 2023-01-24T19:16:30.439 INFO:tasks.rados.rados.0.smithi161.stdout:84: finishing write tid 5 to smithi161112115-84 2023-01-24T19:16:30.439 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 84 v 42 (ObjNum 83 snap 0 seq_num 83) dirty exists 2023-01-24T19:16:30.440 INFO:tasks.rados.rados.0.smithi161.stdout:84: left oid 84 (ObjNum 83 snap 0 seq_num 83) 2023-01-24T19:16:30.440 INFO:tasks.rados.rados.0.smithi161.stdout:85: finishing write tid 2 to smithi161112115-85 2023-01-24T19:16:30.440 INFO:tasks.rados.rados.0.smithi161.stdout:84: done (15 left) 2023-01-24T19:16:30.440 INFO:tasks.rados.rados.0.smithi161.stdout:100: write initial oid 100 2023-01-24T19:16:30.440 INFO:tasks.rados.rados.0.smithi161.stdout:100: seq_num 99 ranges {463893=562240,1593482=658027,2685935=559341} 2023-01-24T19:16:30.443 INFO:tasks.rados.rados.0.smithi161.stdout:100: writing smithi161112115-100 from 463893 to 1026133 tid 1 2023-01-24T19:16:30.450 INFO:tasks.rados.rados.0.smithi161.stdout:100: writing smithi161112115-100 from 1593482 to 2251509 tid 2 2023-01-24T19:16:30.456 INFO:tasks.rados.rados.0.smithi161.stdout:100: writing smithi161112115-100 from 2685935 to 3245276 tid 3 2023-01-24T19:16:30.457 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.457 INFO:tasks.rados.rados.0.smithi161.stdout:85: finishing write tid 3 to smithi161112115-85 2023-01-24T19:16:30.457 INFO:tasks.rados.rados.0.smithi161.stdout:85: finishing write tid 4 to smithi161112115-85 2023-01-24T19:16:30.457 INFO:tasks.rados.rados.0.smithi161.stdout:85: finishing write tid 5 to smithi161112115-85 2023-01-24T19:16:30.457 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 85 v 77 (ObjNum 84 snap 0 seq_num 84) dirty exists 2023-01-24T19:16:30.458 INFO:tasks.rados.rados.0.smithi161.stdout:85: left oid 85 (ObjNum 84 snap 0 seq_num 84) 2023-01-24T19:16:30.458 INFO:tasks.rados.rados.0.smithi161.stdout:86: finishing write tid 1 to smithi161112115-86 2023-01-24T19:16:30.458 INFO:tasks.rados.rados.0.smithi161.stdout:86: finishing write tid 2 to smithi161112115-86 2023-01-24T19:16:30.458 INFO:tasks.rados.rados.0.smithi161.stdout:86: finishing write tid 3 to smithi161112115-86 2023-01-24T19:16:30.458 INFO:tasks.rados.rados.0.smithi161.stdout:85: done (15 left) 2023-01-24T19:16:30.459 INFO:tasks.rados.rados.0.smithi161.stdout:101: write initial oid 101 2023-01-24T19:16:30.459 INFO:tasks.rados.rados.0.smithi161.stdout:101: seq_num 100 ranges {480700=637489,1651964=401685,2713358=770613} 2023-01-24T19:16:30.462 INFO:tasks.rados.rados.0.smithi161.stdout:101: writing smithi161112115-101 from 480700 to 1118189 tid 1 2023-01-24T19:16:30.467 INFO:tasks.rados.rados.0.smithi161.stdout:101: writing smithi161112115-101 from 1651964 to 2053649 tid 2 2023-01-24T19:16:30.476 INFO:tasks.rados.rados.0.smithi161.stdout:101: writing smithi161112115-101 from 2713358 to 3483971 tid 3 2023-01-24T19:16:30.476 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.476 INFO:tasks.rados.rados.0.smithi161.stdout:86: finishing write tid 4 to smithi161112115-86 2023-01-24T19:16:30.477 INFO:tasks.rados.rados.0.smithi161.stdout:86: finishing write tid 5 to smithi161112115-86 2023-01-24T19:16:30.477 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 86 v 35 (ObjNum 85 snap 0 seq_num 85) dirty exists 2023-01-24T19:16:30.477 INFO:tasks.rados.rados.0.smithi161.stdout:86: left oid 86 (ObjNum 85 snap 0 seq_num 85) 2023-01-24T19:16:30.477 INFO:tasks.rados.rados.0.smithi161.stdout:87: finishing write tid 1 to smithi161112115-87 2023-01-24T19:16:30.477 INFO:tasks.rados.rados.0.smithi161.stdout:87: finishing write tid 2 to smithi161112115-87 2023-01-24T19:16:30.477 INFO:tasks.rados.rados.0.smithi161.stdout:87: finishing write tid 3 to smithi161112115-87 2023-01-24T19:16:30.478 INFO:tasks.rados.rados.0.smithi161.stdout:87: finishing write tid 4 to smithi161112115-87 2023-01-24T19:16:30.478 INFO:tasks.rados.rados.0.smithi161.stdout:87: finishing write tid 5 to smithi161112115-87 2023-01-24T19:16:30.478 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 87 v 46 (ObjNum 86 snap 0 seq_num 86) dirty exists 2023-01-24T19:16:30.478 INFO:tasks.rados.rados.0.smithi161.stdout:87: left oid 87 (ObjNum 86 snap 0 seq_num 86) 2023-01-24T19:16:30.478 INFO:tasks.rados.rados.0.smithi161.stdout:88: finishing write tid 1 to smithi161112115-88 2023-01-24T19:16:30.479 INFO:tasks.rados.rados.0.smithi161.stdout:88: finishing write tid 2 to smithi161112115-88 2023-01-24T19:16:30.479 INFO:tasks.rados.rados.0.smithi161.stdout:88: finishing write tid 3 to smithi161112115-88 2023-01-24T19:16:30.479 INFO:tasks.rados.rados.0.smithi161.stdout:86: done (15 left) 2023-01-24T19:16:30.479 INFO:tasks.rados.rados.0.smithi161.stdout:87: done (14 left) 2023-01-24T19:16:30.479 INFO:tasks.rados.rados.0.smithi161.stdout:102: write initial oid 102 2023-01-24T19:16:30.480 INFO:tasks.rados.rados.0.smithi161.stdout:102: seq_num 101 ranges {497507=712738,1710446=661696,3140781=556726} 2023-01-24T19:16:30.483 INFO:tasks.rados.rados.0.smithi161.stdout:102: writing smithi161112115-102 from 497507 to 1210245 tid 1 2023-01-24T19:16:30.489 INFO:tasks.rados.rados.0.smithi161.stdout:102: writing smithi161112115-102 from 1710446 to 2372142 tid 2 2023-01-24T19:16:30.497 INFO:tasks.rados.rados.0.smithi161.stdout:102: writing smithi161112115-102 from 3140781 to 3697507 tid 3 2023-01-24T19:16:30.497 INFO:tasks.rados.rados.0.smithi161.stdout:103: write initial oid 103 2023-01-24T19:16:30.497 INFO:tasks.rados.rados.0.smithi161.stdout:103: seq_num 102 ranges {514314=787987,1768928=405354,2768204=509510} 2023-01-24T19:16:30.504 INFO:tasks.rados.rados.0.smithi161.stdout:103: writing smithi161112115-103 from 514314 to 1302301 tid 1 2023-01-24T19:16:30.509 INFO:tasks.rados.rados.0.smithi161.stdout:103: writing smithi161112115-103 from 1768928 to 2174282 tid 2 2023-01-24T19:16:30.516 INFO:tasks.rados.rados.0.smithi161.stdout:103: writing smithi161112115-103 from 2768204 to 3277714 tid 3 2023-01-24T19:16:30.516 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.516 INFO:tasks.rados.rados.0.smithi161.stdout:88: finishing write tid 4 to smithi161112115-88 2023-01-24T19:16:30.517 INFO:tasks.rados.rados.0.smithi161.stdout:88: finishing write tid 5 to smithi161112115-88 2023-01-24T19:16:30.517 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 88 v 50 (ObjNum 87 snap 0 seq_num 87) dirty exists 2023-01-24T19:16:30.517 INFO:tasks.rados.rados.0.smithi161.stdout:88: left oid 88 (ObjNum 87 snap 0 seq_num 87) 2023-01-24T19:16:30.517 INFO:tasks.rados.rados.0.smithi161.stdout:89: finishing write tid 1 to smithi161112115-89 2023-01-24T19:16:30.517 INFO:tasks.rados.rados.0.smithi161.stdout:88: done (15 left) 2023-01-24T19:16:30.517 INFO:tasks.rados.rados.0.smithi161.stdout:104: write initial oid 104 2023-01-24T19:16:30.518 INFO:tasks.rados.rados.0.smithi161.stdout:104: seq_num 103 ranges {531121=463236,1711057=665365,3079274=651847} 2023-01-24T19:16:30.521 INFO:tasks.rados.rados.0.smithi161.stdout:104: writing smithi161112115-104 from 531121 to 994357 tid 1 2023-01-24T19:16:30.529 INFO:tasks.rados.rados.0.smithi161.stdout:104: writing smithi161112115-104 from 1711057 to 2376422 tid 2 2023-01-24T19:16:30.537 INFO:tasks.rados.rados.0.smithi161.stdout:104: writing smithi161112115-104 from 3079274 to 3731121 tid 3 2023-01-24T19:16:30.537 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.537 INFO:tasks.rados.rados.0.smithi161.stdout:89: finishing write tid 2 to smithi161112115-89 2023-01-24T19:16:30.537 INFO:tasks.rados.rados.0.smithi161.stdout:89: finishing write tid 3 to smithi161112115-89 2023-01-24T19:16:30.537 INFO:tasks.rados.rados.0.smithi161.stdout:89: finishing write tid 4 to smithi161112115-89 2023-01-24T19:16:30.538 INFO:tasks.rados.rados.0.smithi161.stdout:89: finishing write tid 5 to smithi161112115-89 2023-01-24T19:16:30.538 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 89 v 33 (ObjNum 88 snap 0 seq_num 88) dirty exists 2023-01-24T19:16:30.538 INFO:tasks.rados.rados.0.smithi161.stdout:89: left oid 89 (ObjNum 88 snap 0 seq_num 88) 2023-01-24T19:16:30.538 INFO:tasks.rados.rados.0.smithi161.stdout:90: finishing write tid 1 to smithi161112115-90 2023-01-24T19:16:30.538 INFO:tasks.rados.rados.0.smithi161.stdout:90: finishing write tid 2 to smithi161112115-90 2023-01-24T19:16:30.538 INFO:tasks.rados.rados.0.smithi161.stdout:90: finishing write tid 3 to smithi161112115-90 2023-01-24T19:16:30.539 INFO:tasks.rados.rados.0.smithi161.stdout:90: finishing write tid 4 to smithi161112115-90 2023-01-24T19:16:30.539 INFO:tasks.rados.rados.0.smithi161.stdout:90: finishing write tid 5 to smithi161112115-90 2023-01-24T19:16:30.539 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 90 v 39 (ObjNum 89 snap 0 seq_num 89) dirty exists 2023-01-24T19:16:30.539 INFO:tasks.rados.rados.0.smithi161.stdout:90: left oid 90 (ObjNum 89 snap 0 seq_num 89) 2023-01-24T19:16:30.539 INFO:tasks.rados.rados.0.smithi161.stdout:91: finishing write tid 1 to smithi161112115-91 2023-01-24T19:16:30.539 INFO:tasks.rados.rados.0.smithi161.stdout:91: finishing write tid 2 to smithi161112115-91 2023-01-24T19:16:30.540 INFO:tasks.rados.rados.0.smithi161.stdout:89: done (15 left) 2023-01-24T19:16:30.540 INFO:tasks.rados.rados.0.smithi161.stdout:90: done (14 left) 2023-01-24T19:16:30.540 INFO:tasks.rados.rados.0.smithi161.stdout:105: write initial oid 105 2023-01-24T19:16:30.540 INFO:tasks.rados.rados.0.smithi161.stdout:105: seq_num 104 ranges {547928=538485,1769539=409023,2706697=532054} 2023-01-24T19:16:30.542 INFO:tasks.rados.rados.0.smithi161.stdout:105: writing smithi161112115-105 from 547928 to 1086413 tid 1 2023-01-24T19:16:30.548 INFO:tasks.rados.rados.0.smithi161.stdout:105: writing smithi161112115-105 from 1769539 to 2178562 tid 2 2023-01-24T19:16:30.554 INFO:tasks.rados.rados.0.smithi161.stdout:105: writing smithi161112115-105 from 2706697 to 3238751 tid 3 2023-01-24T19:16:30.554 INFO:tasks.rados.rados.0.smithi161.stdout:106: write initial oid 106 2023-01-24T19:16:30.555 INFO:tasks.rados.rados.0.smithi161.stdout:106: seq_num 105 ranges {564735=613734,1828021=669034,3134120=630615} 2023-01-24T19:16:30.561 INFO:tasks.rados.rados.0.smithi161.stdout:106: writing smithi161112115-106 from 564735 to 1178469 tid 1 2023-01-24T19:16:30.568 INFO:tasks.rados.rados.0.smithi161.stdout:106: writing smithi161112115-106 from 1828021 to 2497055 tid 2 2023-01-24T19:16:30.578 INFO:tasks.rados.rados.0.smithi161.stdout:106: writing smithi161112115-106 from 3134120 to 3764735 tid 3 2023-01-24T19:16:30.578 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.579 INFO:tasks.rados.rados.0.smithi161.stdout:91: finishing write tid 3 to smithi161112115-91 2023-01-24T19:16:30.579 INFO:tasks.rados.rados.0.smithi161.stdout:91: finishing write tid 4 to smithi161112115-91 2023-01-24T19:16:30.579 INFO:tasks.rados.rados.0.smithi161.stdout:91: finishing write tid 5 to smithi161112115-91 2023-01-24T19:16:30.579 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 91 v 37 (ObjNum 90 snap 0 seq_num 90) dirty exists 2023-01-24T19:16:30.579 INFO:tasks.rados.rados.0.smithi161.stdout:91: left oid 91 (ObjNum 90 snap 0 seq_num 90) 2023-01-24T19:16:30.579 INFO:tasks.rados.rados.0.smithi161.stdout:92: finishing write tid 1 to smithi161112115-92 2023-01-24T19:16:30.580 INFO:tasks.rados.rados.0.smithi161.stdout:92: finishing write tid 2 to smithi161112115-92 2023-01-24T19:16:30.580 INFO:tasks.rados.rados.0.smithi161.stdout:92: finishing write tid 3 to smithi161112115-92 2023-01-24T19:16:30.580 INFO:tasks.rados.rados.0.smithi161.stdout:92: finishing write tid 4 to smithi161112115-92 2023-01-24T19:16:30.580 INFO:tasks.rados.rados.0.smithi161.stdout:92: finishing write tid 5 to smithi161112115-92 2023-01-24T19:16:30.580 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 92 v 81 (ObjNum 91 snap 0 seq_num 91) dirty exists 2023-01-24T19:16:30.581 INFO:tasks.rados.rados.0.smithi161.stdout:92: left oid 92 (ObjNum 91 snap 0 seq_num 91) 2023-01-24T19:16:30.581 INFO:tasks.rados.rados.0.smithi161.stdout:93: finishing write tid 1 to smithi161112115-93 2023-01-24T19:16:30.581 INFO:tasks.rados.rados.0.smithi161.stdout:93: finishing write tid 2 to smithi161112115-93 2023-01-24T19:16:30.582 INFO:tasks.rados.rados.0.smithi161.stdout:93: finishing write tid 3 to smithi161112115-93 2023-01-24T19:16:30.582 INFO:tasks.rados.rados.0.smithi161.stdout:93: finishing write tid 4 to smithi161112115-93 2023-01-24T19:16:30.582 INFO:tasks.rados.rados.0.smithi161.stdout:93: finishing write tid 5 to smithi161112115-93 2023-01-24T19:16:30.583 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 93 v 37 (ObjNum 92 snap 0 seq_num 92) dirty exists 2023-01-24T19:16:30.583 INFO:tasks.rados.rados.0.smithi161.stdout:93: left oid 93 (ObjNum 92 snap 0 seq_num 92) 2023-01-24T19:16:30.583 INFO:tasks.rados.rados.0.smithi161.stdout:94: finishing write tid 1 to smithi161112115-94 2023-01-24T19:16:30.583 INFO:tasks.rados.rados.0.smithi161.stdout:94: finishing write tid 2 to smithi161112115-94 2023-01-24T19:16:30.584 INFO:tasks.rados.rados.0.smithi161.stdout:94: finishing write tid 3 to smithi161112115-94 2023-01-24T19:16:30.584 INFO:tasks.rados.rados.0.smithi161.stdout:94: finishing write tid 4 to smithi161112115-94 2023-01-24T19:16:30.584 INFO:tasks.rados.rados.0.smithi161.stdout:94: finishing write tid 5 to smithi161112115-94 2023-01-24T19:16:30.584 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 94 v 41 (ObjNum 93 snap 0 seq_num 93) dirty exists 2023-01-24T19:16:30.584 INFO:tasks.rados.rados.0.smithi161.stdout:94: left oid 94 (ObjNum 93 snap 0 seq_num 93) 2023-01-24T19:16:30.585 INFO:tasks.rados.rados.0.smithi161.stdout:95: finishing write tid 1 to smithi161112115-95 2023-01-24T19:16:30.585 INFO:tasks.rados.rados.0.smithi161.stdout:95: finishing write tid 2 to smithi161112115-95 2023-01-24T19:16:30.585 INFO:tasks.rados.rados.0.smithi161.stdout:95: finishing write tid 3 to smithi161112115-95 2023-01-24T19:16:30.585 INFO:tasks.rados.rados.0.smithi161.stdout:95: finishing write tid 4 to smithi161112115-95 2023-01-24T19:16:30.586 INFO:tasks.rados.rados.0.smithi161.stdout:95: finishing write tid 5 to smithi161112115-95 2023-01-24T19:16:30.586 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 95 v 41 (ObjNum 94 snap 0 seq_num 94) dirty exists 2023-01-24T19:16:30.586 INFO:tasks.rados.rados.0.smithi161.stdout:95: left oid 95 (ObjNum 94 snap 0 seq_num 94) 2023-01-24T19:16:30.586 INFO:tasks.rados.rados.0.smithi161.stdout:96: finishing write tid 1 to smithi161112115-96 2023-01-24T19:16:30.586 INFO:tasks.rados.rados.0.smithi161.stdout:96: finishing write tid 2 to smithi161112115-96 2023-01-24T19:16:30.587 INFO:tasks.rados.rados.0.smithi161.stdout:96: finishing write tid 3 to smithi161112115-96 2023-01-24T19:16:30.587 INFO:tasks.rados.rados.0.smithi161.stdout:96: finishing write tid 4 to smithi161112115-96 2023-01-24T19:16:30.587 INFO:tasks.rados.rados.0.smithi161.stdout:96: finishing write tid 5 to smithi161112115-96 2023-01-24T19:16:30.587 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 96 v 45 (ObjNum 95 snap 0 seq_num 95) dirty exists 2023-01-24T19:16:30.587 INFO:tasks.rados.rados.0.smithi161.stdout:96: left oid 96 (ObjNum 95 snap 0 seq_num 95) 2023-01-24T19:16:30.588 INFO:tasks.rados.rados.0.smithi161.stdout:97: finishing write tid 1 to smithi161112115-97 2023-01-24T19:16:30.588 INFO:tasks.rados.rados.0.smithi161.stdout:97: finishing write tid 2 to smithi161112115-97 2023-01-24T19:16:30.588 INFO:tasks.rados.rados.0.smithi161.stdout:97: finishing write tid 3 to smithi161112115-97 2023-01-24T19:16:30.588 INFO:tasks.rados.rados.0.smithi161.stdout:97: finishing write tid 4 to smithi161112115-97 2023-01-24T19:16:30.588 INFO:tasks.rados.rados.0.smithi161.stdout:97: finishing write tid 5 to smithi161112115-97 2023-01-24T19:16:30.589 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 97 v 45 (ObjNum 96 snap 0 seq_num 96) dirty exists 2023-01-24T19:16:30.589 INFO:tasks.rados.rados.0.smithi161.stdout:97: left oid 97 (ObjNum 96 snap 0 seq_num 96) 2023-01-24T19:16:30.589 INFO:tasks.rados.rados.0.smithi161.stdout:98: finishing write tid 1 to smithi161112115-98 2023-01-24T19:16:30.589 INFO:tasks.rados.rados.0.smithi161.stdout:98: finishing write tid 2 to smithi161112115-98 2023-01-24T19:16:30.590 INFO:tasks.rados.rados.0.smithi161.stdout:98: finishing write tid 3 to smithi161112115-98 2023-01-24T19:16:30.590 INFO:tasks.rados.rados.0.smithi161.stdout:98: finishing write tid 4 to smithi161112115-98 2023-01-24T19:16:30.590 INFO:tasks.rados.rados.0.smithi161.stdout:98: finishing write tid 5 to smithi161112115-98 2023-01-24T19:16:30.590 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 98 v 43 (ObjNum 97 snap 0 seq_num 97) dirty exists 2023-01-24T19:16:30.590 INFO:tasks.rados.rados.0.smithi161.stdout:98: left oid 98 (ObjNum 97 snap 0 seq_num 97) 2023-01-24T19:16:30.591 INFO:tasks.rados.rados.0.smithi161.stdout:99: finishing write tid 1 to smithi161112115-99 2023-01-24T19:16:30.591 INFO:tasks.rados.rados.0.smithi161.stdout:99: finishing write tid 2 to smithi161112115-99 2023-01-24T19:16:30.591 INFO:tasks.rados.rados.0.smithi161.stdout:99: finishing write tid 3 to smithi161112115-99 2023-01-24T19:16:30.591 INFO:tasks.rados.rados.0.smithi161.stdout:99: finishing write tid 4 to smithi161112115-99 2023-01-24T19:16:30.591 INFO:tasks.rados.rados.0.smithi161.stdout:99: finishing write tid 5 to smithi161112115-99 2023-01-24T19:16:30.592 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 99 v 49 (ObjNum 98 snap 0 seq_num 98) dirty exists 2023-01-24T19:16:30.592 INFO:tasks.rados.rados.0.smithi161.stdout:99: left oid 99 (ObjNum 98 snap 0 seq_num 98) 2023-01-24T19:16:30.592 INFO:tasks.rados.rados.0.smithi161.stdout:100: finishing write tid 1 to smithi161112115-100 2023-01-24T19:16:30.592 INFO:tasks.rados.rados.0.smithi161.stdout:100: finishing write tid 2 to smithi161112115-100 2023-01-24T19:16:30.592 INFO:tasks.rados.rados.0.smithi161.stdout:100: finishing write tid 3 to smithi161112115-100 2023-01-24T19:16:30.593 INFO:tasks.rados.rados.0.smithi161.stdout:101: finishing write tid 1 to smithi161112115-101 2023-01-24T19:16:30.593 INFO:tasks.rados.rados.0.smithi161.stdout:101: finishing write tid 2 to smithi161112115-101 2023-01-24T19:16:30.593 INFO:tasks.rados.rados.0.smithi161.stdout:100: finishing write tid 4 to smithi161112115-100 2023-01-24T19:16:30.593 INFO:tasks.rados.rados.0.smithi161.stdout:100: finishing write tid 5 to smithi161112115-100 2023-01-24T19:16:30.594 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 100 v 49 (ObjNum 99 snap 0 seq_num 99) dirty exists 2023-01-24T19:16:30.594 INFO:tasks.rados.rados.0.smithi161.stdout:100: left oid 100 (ObjNum 99 snap 0 seq_num 99) 2023-01-24T19:16:30.594 INFO:tasks.rados.rados.0.smithi161.stdout:101: finishing write tid 3 to smithi161112115-101 2023-01-24T19:16:30.594 INFO:tasks.rados.rados.0.smithi161.stdout:101: finishing write tid 4 to smithi161112115-101 2023-01-24T19:16:30.594 INFO:tasks.rados.rados.0.smithi161.stdout:101: finishing write tid 5 to smithi161112115-101 2023-01-24T19:16:30.595 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 101 v 85 (ObjNum 100 snap 0 seq_num 100) dirty exists 2023-01-24T19:16:30.595 INFO:tasks.rados.rados.0.smithi161.stdout:101: left oid 101 (ObjNum 100 snap 0 seq_num 100) 2023-01-24T19:16:30.595 INFO:tasks.rados.rados.0.smithi161.stdout:102: finishing write tid 1 to smithi161112115-102 2023-01-24T19:16:30.595 INFO:tasks.rados.rados.0.smithi161.stdout:102: finishing write tid 2 to smithi161112115-102 2023-01-24T19:16:30.595 INFO:tasks.rados.rados.0.smithi161.stdout:102: finishing write tid 3 to smithi161112115-102 2023-01-24T19:16:30.596 INFO:tasks.rados.rados.0.smithi161.stdout:102: finishing write tid 4 to smithi161112115-102 2023-01-24T19:16:30.596 INFO:tasks.rados.rados.0.smithi161.stdout:102: finishing write tid 5 to smithi161112115-102 2023-01-24T19:16:30.596 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 102 v 47 (ObjNum 101 snap 0 seq_num 101) dirty exists 2023-01-24T19:16:30.596 INFO:tasks.rados.rados.0.smithi161.stdout:102: left oid 102 (ObjNum 101 snap 0 seq_num 101) 2023-01-24T19:16:30.597 INFO:tasks.rados.rados.0.smithi161.stdout:103: finishing write tid 1 to smithi161112115-103 2023-01-24T19:16:30.597 INFO:tasks.rados.rados.0.smithi161.stdout:103: finishing write tid 2 to smithi161112115-103 2023-01-24T19:16:30.597 INFO:tasks.rados.rados.0.smithi161.stdout:103: finishing write tid 3 to smithi161112115-103 2023-01-24T19:16:30.597 INFO:tasks.rados.rados.0.smithi161.stdout:103: finishing write tid 4 to smithi161112115-103 2023-01-24T19:16:30.597 INFO:tasks.rados.rados.0.smithi161.stdout:103: finishing write tid 5 to smithi161112115-103 2023-01-24T19:16:30.598 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 103 v 89 (ObjNum 102 snap 0 seq_num 102) dirty exists 2023-01-24T19:16:30.598 INFO:tasks.rados.rados.0.smithi161.stdout:103: left oid 103 (ObjNum 102 snap 0 seq_num 102) 2023-01-24T19:16:30.598 INFO:tasks.rados.rados.0.smithi161.stdout:104: finishing write tid 1 to smithi161112115-104 2023-01-24T19:16:30.598 INFO:tasks.rados.rados.0.smithi161.stdout:104: finishing write tid 2 to smithi161112115-104 2023-01-24T19:16:30.598 INFO:tasks.rados.rados.0.smithi161.stdout:104: finishing write tid 3 to smithi161112115-104 2023-01-24T19:16:30.599 INFO:tasks.rados.rados.0.smithi161.stdout:104: finishing write tid 4 to smithi161112115-104 2023-01-24T19:16:30.599 INFO:tasks.rados.rados.0.smithi161.stdout:104: finishing write tid 5 to smithi161112115-104 2023-01-24T19:16:30.599 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 104 v 53 (ObjNum 103 snap 0 seq_num 103) dirty exists 2023-01-24T19:16:30.599 INFO:tasks.rados.rados.0.smithi161.stdout:104: left oid 104 (ObjNum 103 snap 0 seq_num 103) 2023-01-24T19:16:30.599 INFO:tasks.rados.rados.0.smithi161.stdout:105: finishing write tid 1 to smithi161112115-105 2023-01-24T19:16:30.600 INFO:tasks.rados.rados.0.smithi161.stdout:105: finishing write tid 2 to smithi161112115-105 2023-01-24T19:16:30.600 INFO:tasks.rados.rados.0.smithi161.stdout:105: finishing write tid 3 to smithi161112115-105 2023-01-24T19:16:30.600 INFO:tasks.rados.rados.0.smithi161.stdout:106: finishing write tid 1 to smithi161112115-106 2023-01-24T19:16:30.600 INFO:tasks.rados.rados.0.smithi161.stdout:105: finishing write tid 4 to smithi161112115-105 2023-01-24T19:16:30.600 INFO:tasks.rados.rados.0.smithi161.stdout:105: finishing write tid 5 to smithi161112115-105 2023-01-24T19:16:30.601 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 105 v 52 (ObjNum 104 snap 0 seq_num 104) dirty exists 2023-01-24T19:16:30.601 INFO:tasks.rados.rados.0.smithi161.stdout:105: left oid 105 (ObjNum 104 snap 0 seq_num 104) 2023-01-24T19:16:30.601 INFO:tasks.rados.rados.0.smithi161.stdout:106: finishing write tid 2 to smithi161112115-106 2023-01-24T19:16:30.601 INFO:tasks.rados.rados.0.smithi161.stdout:91: done (15 left) 2023-01-24T19:16:30.602 INFO:tasks.rados.rados.0.smithi161.stdout:92: done (14 left) 2023-01-24T19:16:30.602 INFO:tasks.rados.rados.0.smithi161.stdout:93: done (13 left) 2023-01-24T19:16:30.602 INFO:tasks.rados.rados.0.smithi161.stdout:94: done (12 left) 2023-01-24T19:16:30.602 INFO:tasks.rados.rados.0.smithi161.stdout:95: done (11 left) 2023-01-24T19:16:30.602 INFO:tasks.rados.rados.0.smithi161.stdout:96: done (10 left) 2023-01-24T19:16:30.603 INFO:tasks.rados.rados.0.smithi161.stdout:97: done (9 left) 2023-01-24T19:16:30.603 INFO:tasks.rados.rados.0.smithi161.stdout:98: done (8 left) 2023-01-24T19:16:30.603 INFO:tasks.rados.rados.0.smithi161.stdout:99: done (7 left) 2023-01-24T19:16:30.603 INFO:tasks.rados.rados.0.smithi161.stdout:100: done (6 left) 2023-01-24T19:16:30.603 INFO:tasks.rados.rados.0.smithi161.stdout:101: done (5 left) 2023-01-24T19:16:30.604 INFO:tasks.rados.rados.0.smithi161.stdout:102: done (4 left) 2023-01-24T19:16:30.604 INFO:tasks.rados.rados.0.smithi161.stdout:103: done (3 left) 2023-01-24T19:16:30.604 INFO:tasks.rados.rados.0.smithi161.stdout:104: done (2 left) 2023-01-24T19:16:30.604 INFO:tasks.rados.rados.0.smithi161.stdout:105: done (1 left) 2023-01-24T19:16:30.605 INFO:tasks.rados.rados.0.smithi161.stdout:107: write initial oid 107 2023-01-24T19:16:30.605 INFO:tasks.rados.rados.0.smithi161.stdout:107: seq_num 106 ranges {581542=688983,1886503=412692,2761543=670951} 2023-01-24T19:16:30.606 INFO:tasks.rados.rados.0.smithi161.stdout:107: writing smithi161112115-107 from 581542 to 1270525 tid 1 2023-01-24T19:16:30.606 INFO:tasks.rados.rados.0.smithi161.stdout:107: writing smithi161112115-107 from 1886503 to 2299195 tid 2 2023-01-24T19:16:30.607 INFO:tasks.rados.rados.0.smithi161.stdout:107: writing smithi161112115-107 from 2761543 to 3432494 tid 3 2023-01-24T19:16:30.607 INFO:tasks.rados.rados.0.smithi161.stdout:108: write initial oid 108 2023-01-24T19:16:30.607 INFO:tasks.rados.rados.0.smithi161.stdout:106: finishing write tid 3 to smithi161112115-106 2023-01-24T19:16:30.607 INFO:tasks.rados.rados.0.smithi161.stdout:106: finishing write tid 4 to smithi161112115-106 2023-01-24T19:16:30.607 INFO:tasks.rados.rados.0.smithi161.stdout:106: finishing write tid 5 to smithi161112115-106 2023-01-24T19:16:30.608 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 106 v 51 (ObjNum 105 snap 0 seq_num 105) dirty exists 2023-01-24T19:16:30.608 INFO:tasks.rados.rados.0.smithi161.stdout:106: left oid 106 (ObjNum 105 snap 0 seq_num 105) 2023-01-24T19:16:30.608 INFO:tasks.rados.rados.0.smithi161.stdout:107: finishing write tid 1 to smithi161112115-107 2023-01-24T19:16:30.608 INFO:tasks.rados.rados.0.smithi161.stdout:108: seq_num 107 ranges {598349=480585,1544985=672703,2905319=482223} 2023-01-24T19:16:30.609 INFO:tasks.rados.rados.0.smithi161.stdout:108: writing smithi161112115-108 from 598349 to 1078934 tid 1 2023-01-24T19:16:30.609 INFO:tasks.rados.rados.0.smithi161.stdout:108: writing smithi161112115-108 from 1544985 to 2217688 tid 2 2023-01-24T19:16:30.614 INFO:tasks.rados.rados.0.smithi161.stdout:108: writing smithi161112115-108 from 2905319 to 3387542 tid 3 2023-01-24T19:16:30.615 INFO:tasks.rados.rados.0.smithi161.stdout:106: done (2 left) 2023-01-24T19:16:30.615 INFO:tasks.rados.rados.0.smithi161.stdout:109: write initial oid 109 2023-01-24T19:16:30.615 INFO:tasks.rados.rados.0.smithi161.stdout:109: seq_num 108 ranges {615156=555834,1603467=416361,2816389=693495} 2023-01-24T19:16:30.621 INFO:tasks.rados.rados.0.smithi161.stdout:109: writing smithi161112115-109 from 615156 to 1170990 tid 1 2023-01-24T19:16:30.626 INFO:tasks.rados.rados.0.smithi161.stdout:109: writing smithi161112115-109 from 1603467 to 2019828 tid 2 2023-01-24T19:16:30.634 INFO:tasks.rados.rados.0.smithi161.stdout:109: writing smithi161112115-109 from 2816389 to 3509884 tid 3 2023-01-24T19:16:30.635 INFO:tasks.rados.rados.0.smithi161.stdout:110: write initial oid 110 2023-01-24T19:16:30.635 INFO:tasks.rados.rados.0.smithi161.stdout:107: finishing write tid 2 to smithi161112115-107 2023-01-24T19:16:30.635 INFO:tasks.rados.rados.0.smithi161.stdout:110: seq_num 109 ranges {631963=631083,2061949=560019,3243812=504767} 2023-01-24T19:16:30.642 INFO:tasks.rados.rados.0.smithi161.stdout:110: writing smithi161112115-110 from 631963 to 1263046 tid 1 2023-01-24T19:16:30.649 INFO:tasks.rados.rados.0.smithi161.stdout:110: writing smithi161112115-110 from 2061949 to 2621968 tid 2 2023-01-24T19:16:30.656 INFO:tasks.rados.rados.0.smithi161.stdout:110: writing smithi161112115-110 from 3243812 to 3748579 tid 3 2023-01-24T19:16:30.656 INFO:tasks.rados.rados.0.smithi161.stdout:111: write initial oid 111 2023-01-24T19:16:30.656 INFO:tasks.rados.rados.0.smithi161.stdout:107: finishing write tid 3 to smithi161112115-107 2023-01-24T19:16:30.656 INFO:tasks.rados.rados.0.smithi161.stdout:108: finishing write tid 1 to smithi161112115-108 2023-01-24T19:16:30.657 INFO:tasks.rados.rados.0.smithi161.stdout:111: seq_num 110 ranges {648770=706332,2120431=420030,3271235=432392} 2023-01-24T19:16:30.663 INFO:tasks.rados.rados.0.smithi161.stdout:111: writing smithi161112115-111 from 648770 to 1355102 tid 1 2023-01-24T19:16:30.670 INFO:tasks.rados.rados.0.smithi161.stdout:111: writing smithi161112115-111 from 2120431 to 2540461 tid 2 2023-01-24T19:16:30.676 INFO:tasks.rados.rados.0.smithi161.stdout:111: writing smithi161112115-111 from 3271235 to 3703627 tid 3 2023-01-24T19:16:30.677 INFO:tasks.rados.rados.0.smithi161.stdout:112: write initial oid 112 2023-01-24T19:16:30.677 INFO:tasks.rados.rados.0.smithi161.stdout:107: finishing write tid 4 to smithi161112115-107 2023-01-24T19:16:30.677 INFO:tasks.rados.rados.0.smithi161.stdout:107: finishing write tid 5 to smithi161112115-107 2023-01-24T19:16:30.677 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 107 v 90 (ObjNum 106 snap 0 seq_num 106) dirty exists 2023-01-24T19:16:30.678 INFO:tasks.rados.rados.0.smithi161.stdout:107: left oid 107 (ObjNum 106 snap 0 seq_num 106) 2023-01-24T19:16:30.678 INFO:tasks.rados.rados.0.smithi161.stdout:108: finishing write tid 2 to smithi161112115-108 2023-01-24T19:16:30.678 INFO:tasks.rados.rados.0.smithi161.stdout:112: seq_num 111 ranges {665577=781581,2062560=563688,3182305=643664} 2023-01-24T19:16:30.684 INFO:tasks.rados.rados.0.smithi161.stdout:112: writing smithi161112115-112 from 665577 to 1447158 tid 1 2023-01-24T19:16:30.691 INFO:tasks.rados.rados.0.smithi161.stdout:112: writing smithi161112115-112 from 2062560 to 2626248 tid 2 2023-01-24T19:16:30.697 INFO:tasks.rados.rados.0.smithi161.stdout:112: writing smithi161112115-112 from 3182305 to 3825969 tid 3 2023-01-24T19:16:30.698 INFO:tasks.rados.rados.0.smithi161.stdout:107: done (5 left) 2023-01-24T19:16:30.698 INFO:tasks.rados.rados.0.smithi161.stdout:113: write initial oid 113 2023-01-24T19:16:30.698 INFO:tasks.rados.rados.0.smithi161.stdout:113: seq_num 112 ranges {682384=456830,1721042=423699,2809728=454936,3731357=151027} 2023-01-24T19:16:30.703 INFO:tasks.rados.rados.0.smithi161.stdout:113: writing smithi161112115-113 from 682384 to 1139214 tid 1 2023-01-24T19:16:30.709 INFO:tasks.rados.rados.0.smithi161.stdout:113: writing smithi161112115-113 from 1721042 to 2144741 tid 2 2023-01-24T19:16:30.715 INFO:tasks.rados.rados.0.smithi161.stdout:113: writing smithi161112115-113 from 2809728 to 3264664 tid 3 2023-01-24T19:16:30.719 INFO:tasks.rados.rados.0.smithi161.stdout:113: writing smithi161112115-113 from 3731357 to 3882384 tid 4 2023-01-24T19:16:30.719 INFO:tasks.rados.rados.0.smithi161.stdout:114: write initial oid 114 2023-01-24T19:16:30.719 INFO:tasks.rados.rados.0.smithi161.stdout:114: seq_num 113 ranges {699191=532079,1779524=567357,2837151=666208} 2023-01-24T19:16:30.725 INFO:tasks.rados.rados.0.smithi161.stdout:114: writing smithi161112115-114 from 699191 to 1231270 tid 1 2023-01-24T19:16:30.731 INFO:tasks.rados.rados.0.smithi161.stdout:114: writing smithi161112115-114 from 1779524 to 2346881 tid 2 2023-01-24T19:16:30.738 INFO:tasks.rados.rados.0.smithi161.stdout:114: writing smithi161112115-114 from 2837151 to 3503359 tid 3 2023-01-24T19:16:30.738 INFO:tasks.rados.rados.0.smithi161.stdout:115: write initial oid 115 2023-01-24T19:16:30.739 INFO:tasks.rados.rados.0.smithi161.stdout:115: seq_num 114 ranges {715998=607328,1838006=427368,2864574=477480,3863835=52163} 2023-01-24T19:16:30.745 INFO:tasks.rados.rados.0.smithi161.stdout:115: writing smithi161112115-115 from 715998 to 1323326 tid 1 2023-01-24T19:16:30.751 INFO:tasks.rados.rados.0.smithi161.stdout:115: writing smithi161112115-115 from 1838006 to 2265374 tid 2 2023-01-24T19:16:30.757 INFO:tasks.rados.rados.0.smithi161.stdout:115: writing smithi161112115-115 from 2864574 to 3342054 tid 3 2023-01-24T19:16:30.761 INFO:tasks.rados.rados.0.smithi161.stdout:115: writing smithi161112115-115 from 3863835 to 3915998 tid 4 2023-01-24T19:16:30.761 INFO:tasks.rados.rados.0.smithi161.stdout:116: write initial oid 116 2023-01-24T19:16:30.761 INFO:tasks.rados.rados.0.smithi161.stdout:108: finishing write tid 3 to smithi161112115-108 2023-01-24T19:16:30.761 INFO:tasks.rados.rados.0.smithi161.stdout:108: finishing write tid 4 to smithi161112115-108 2023-01-24T19:16:30.761 INFO:tasks.rados.rados.0.smithi161.stdout:108: finishing write tid 5 to smithi161112115-108 2023-01-24T19:16:30.762 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 108 v 55 (ObjNum 107 snap 0 seq_num 107) dirty exists 2023-01-24T19:16:30.762 INFO:tasks.rados.rados.0.smithi161.stdout:108: left oid 108 (ObjNum 107 snap 0 seq_num 107) 2023-01-24T19:16:30.762 INFO:tasks.rados.rados.0.smithi161.stdout:109: finishing write tid 1 to smithi161112115-109 2023-01-24T19:16:30.762 INFO:tasks.rados.rados.0.smithi161.stdout:109: finishing write tid 2 to smithi161112115-109 2023-01-24T19:16:30.762 INFO:tasks.rados.rados.0.smithi161.stdout:116: seq_num 115 ranges {732805=798930,2296488=571026,3291997=405105} 2023-01-24T19:16:30.769 INFO:tasks.rados.rados.0.smithi161.stdout:116: writing smithi161112115-116 from 732805 to 1531735 tid 1 2023-01-24T19:16:30.776 INFO:tasks.rados.rados.0.smithi161.stdout:116: writing smithi161112115-116 from 2296488 to 2867514 tid 2 2023-01-24T19:16:30.781 INFO:tasks.rados.rados.0.smithi161.stdout:116: writing smithi161112115-116 from 3291997 to 3697102 tid 3 2023-01-24T19:16:30.782 INFO:tasks.rados.rados.0.smithi161.stdout:108: done (8 left) 2023-01-24T19:16:30.782 INFO:tasks.rados.rados.0.smithi161.stdout:117: write initial oid 117 2023-01-24T19:16:30.782 INFO:tasks.rados.rados.0.smithi161.stdout:117: seq_num 116 ranges {749612=474179,1954970=431037,2919420=616377} 2023-01-24T19:16:30.788 INFO:tasks.rados.rados.0.smithi161.stdout:117: writing smithi161112115-117 from 749612 to 1223791 tid 1 2023-01-24T19:16:30.794 INFO:tasks.rados.rados.0.smithi161.stdout:117: writing smithi161112115-117 from 1954970 to 2386007 tid 2 2023-01-24T19:16:30.801 INFO:tasks.rados.rados.0.smithi161.stdout:117: writing smithi161112115-117 from 2919420 to 3535797 tid 3 2023-01-24T19:16:30.801 INFO:tasks.rados.rados.0.smithi161.stdout:118: write initial oid 118 2023-01-24T19:16:30.801 INFO:tasks.rados.rados.0.smithi161.stdout:118: seq_num 117 ranges {766419=549428,2013452=574695,3346843=427649} 2023-01-24T19:16:30.808 INFO:tasks.rados.rados.0.smithi161.stdout:118: writing smithi161112115-118 from 766419 to 1315847 tid 1 2023-01-24T19:16:30.815 INFO:tasks.rados.rados.0.smithi161.stdout:118: writing smithi161112115-118 from 2013452 to 2588147 tid 2 2023-01-24T19:16:30.822 INFO:tasks.rados.rados.0.smithi161.stdout:118: writing smithi161112115-118 from 3346843 to 3774492 tid 3 2023-01-24T19:16:30.822 INFO:tasks.rados.rados.0.smithi161.stdout:119: write initial oid 119 2023-01-24T19:16:30.822 INFO:tasks.rados.rados.0.smithi161.stdout:119: seq_num 118 ranges {783226=624677,2071934=434706,2974266=638921} 2023-01-24T19:16:30.829 INFO:tasks.rados.rados.0.smithi161.stdout:119: writing smithi161112115-119 from 783226 to 1407903 tid 1 2023-01-24T19:16:30.835 INFO:tasks.rados.rados.0.smithi161.stdout:119: writing smithi161112115-119 from 2071934 to 2506640 tid 2 2023-01-24T19:16:30.842 INFO:tasks.rados.rados.0.smithi161.stdout:119: writing smithi161112115-119 from 2974266 to 3613187 tid 3 2023-01-24T19:16:30.842 INFO:tasks.rados.rados.0.smithi161.stdout:120: write initial oid 120 2023-01-24T19:16:30.843 INFO:tasks.rados.rados.0.smithi161.stdout:120: seq_num 119 ranges {400033=699926,1614063=385970} 2023-01-24T19:16:30.848 INFO:tasks.rados.rados.0.smithi161.stdout:120: writing smithi161112115-120 from 400033 to 1099959 tid 1 2023-01-24T19:16:30.853 INFO:tasks.rados.rados.0.smithi161.stdout:120: writing smithi161112115-120 from 1614063 to 2000033 tid 2 2023-01-24T19:16:30.853 INFO:tasks.rados.rados.0.smithi161.stdout:121: write initial oid 121 2023-01-24T19:16:30.854 INFO:tasks.rados.rados.0.smithi161.stdout:121: seq_num 120 ranges {416840=775175,1672545=344295} 2023-01-24T19:16:30.860 INFO:tasks.rados.rados.0.smithi161.stdout:121: writing smithi161112115-121 from 416840 to 1192015 tid 1 2023-01-24T19:16:30.864 INFO:tasks.rados.rados.0.smithi161.stdout:121: writing smithi161112115-121 from 1672545 to 2016840 tid 2 2023-01-24T19:16:30.865 INFO:tasks.rados.rados.0.smithi161.stdout:122: write initial oid 122 2023-01-24T19:16:30.865 INFO:tasks.rados.rados.0.smithi161.stdout:122: seq_num 121 ranges {433647=450424,1331027=582033} 2023-01-24T19:16:30.869 INFO:tasks.rados.rados.0.smithi161.stdout:122: writing smithi161112115-122 from 433647 to 884071 tid 1 2023-01-24T19:16:30.875 INFO:tasks.rados.rados.0.smithi161.stdout:122: writing smithi161112115-122 from 1331027 to 1913060 tid 2 2023-01-24T19:16:30.876 INFO:tasks.rados.rados.0.smithi161.stdout:123: write initial oid 123 2023-01-24T19:16:30.876 INFO:tasks.rados.rados.0.smithi161.stdout:109: finishing write tid 3 to smithi161112115-109 2023-01-24T19:16:30.876 INFO:tasks.rados.rados.0.smithi161.stdout:109: finishing write tid 4 to smithi161112115-109 2023-01-24T19:16:30.876 INFO:tasks.rados.rados.0.smithi161.stdout:123: seq_num 122 ranges {450454=642026,1505862=544592} 2023-01-24T19:16:30.881 INFO:tasks.rados.rados.0.smithi161.stdout:123: writing smithi161112115-123 from 450454 to 1092480 tid 1 2023-01-24T19:16:30.887 INFO:tasks.rados.rados.0.smithi161.stdout:123: writing smithi161112115-123 from 1505862 to 2050454 tid 2 2023-01-24T19:16:30.887 INFO:tasks.rados.rados.0.smithi161.stdout:124: write initial oid 124 2023-01-24T19:16:30.888 INFO:tasks.rados.rados.0.smithi161.stdout:109: finishing write tid 5 to smithi161112115-109 2023-01-24T19:16:30.888 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 109 v 55 (ObjNum 108 snap 0 seq_num 108) dirty exists 2023-01-24T19:16:30.888 INFO:tasks.rados.rados.0.smithi161.stdout:109: left oid 109 (ObjNum 108 snap 0 seq_num 108) 2023-01-24T19:16:30.888 INFO:tasks.rados.rados.0.smithi161.stdout:110: finishing write tid 1 to smithi161112115-110 2023-01-24T19:16:30.888 INFO:tasks.rados.rados.0.smithi161.stdout:110: finishing write tid 2 to smithi161112115-110 2023-01-24T19:16:30.889 INFO:tasks.rados.rados.0.smithi161.stdout:124: seq_num 123 ranges {467261=717275,1847991=219270} 2023-01-24T19:16:30.893 INFO:tasks.rados.rados.0.smithi161.stdout:124: writing smithi161112115-124 from 467261 to 1184536 tid 1 2023-01-24T19:16:30.899 INFO:tasks.rados.rados.0.smithi161.stdout:124: writing smithi161112115-124 from 1847991 to 2067261 tid 2 2023-01-24T19:16:30.899 INFO:tasks.rados.rados.0.smithi161.stdout:109: done (15 left) 2023-01-24T19:16:30.900 INFO:tasks.rados.rados.0.smithi161.stdout:125: write initial oid 125 2023-01-24T19:16:30.900 INFO:tasks.rados.rados.0.smithi161.stdout:125: seq_num 124 ranges {484068=792524,1906473=177595} 2023-01-24T19:16:30.905 INFO:tasks.rados.rados.0.smithi161.stdout:125: writing smithi161112115-125 from 484068 to 1276592 tid 1 2023-01-24T19:16:30.911 INFO:tasks.rados.rados.0.smithi161.stdout:125: writing smithi161112115-125 from 1906473 to 2084068 tid 2 2023-01-24T19:16:30.911 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.911 INFO:tasks.rados.rados.0.smithi161.stdout:110: finishing write tid 3 to smithi161112115-110 2023-01-24T19:16:30.911 INFO:tasks.rados.rados.0.smithi161.stdout:110: finishing write tid 4 to smithi161112115-110 2023-01-24T19:16:30.912 INFO:tasks.rados.rados.0.smithi161.stdout:110: finishing write tid 5 to smithi161112115-110 2023-01-24T19:16:30.912 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 110 v 59 (ObjNum 109 snap 0 seq_num 109) dirty exists 2023-01-24T19:16:30.912 INFO:tasks.rados.rados.0.smithi161.stdout:110: left oid 110 (ObjNum 109 snap 0 seq_num 109) 2023-01-24T19:16:30.912 INFO:tasks.rados.rados.0.smithi161.stdout:111: finishing write tid 1 to smithi161112115-111 2023-01-24T19:16:30.912 INFO:tasks.rados.rados.0.smithi161.stdout:111: finishing write tid 2 to smithi161112115-111 2023-01-24T19:16:30.913 INFO:tasks.rados.rados.0.smithi161.stdout:110: done (15 left) 2023-01-24T19:16:30.913 INFO:tasks.rados.rados.0.smithi161.stdout:126: write initial oid 126 2023-01-24T19:16:30.913 INFO:tasks.rados.rados.0.smithi161.stdout:126: seq_num 125 ranges {500875=467773,1564955=535920} 2023-01-24T19:16:30.915 INFO:tasks.rados.rados.0.smithi161.stdout:126: writing smithi161112115-126 from 500875 to 968648 tid 1 2023-01-24T19:16:30.922 INFO:tasks.rados.rados.0.smithi161.stdout:126: writing smithi161112115-126 from 1564955 to 2100875 tid 2 2023-01-24T19:16:30.923 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.923 INFO:tasks.rados.rados.0.smithi161.stdout:111: finishing write tid 3 to smithi161112115-111 2023-01-24T19:16:30.923 INFO:tasks.rados.rados.0.smithi161.stdout:111: finishing write tid 4 to smithi161112115-111 2023-01-24T19:16:30.923 INFO:tasks.rados.rados.0.smithi161.stdout:111: finishing write tid 5 to smithi161112115-111 2023-01-24T19:16:30.923 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 111 v 93 (ObjNum 110 snap 0 seq_num 110) dirty exists 2023-01-24T19:16:30.924 INFO:tasks.rados.rados.0.smithi161.stdout:111: left oid 111 (ObjNum 110 snap 0 seq_num 110) 2023-01-24T19:16:30.924 INFO:tasks.rados.rados.0.smithi161.stdout:112: finishing write tid 1 to smithi161112115-112 2023-01-24T19:16:30.924 INFO:tasks.rados.rados.0.smithi161.stdout:112: finishing write tid 2 to smithi161112115-112 2023-01-24T19:16:30.924 INFO:tasks.rados.rados.0.smithi161.stdout:111: done (15 left) 2023-01-24T19:16:30.924 INFO:tasks.rados.rados.0.smithi161.stdout:127: write initial oid 127 2023-01-24T19:16:30.925 INFO:tasks.rados.rados.0.smithi161.stdout:127: seq_num 126 ranges {517682=543022,1623437=494245} 2023-01-24T19:16:30.928 INFO:tasks.rados.rados.0.smithi161.stdout:127: writing smithi161112115-127 from 517682 to 1060704 tid 1 2023-01-24T19:16:30.935 INFO:tasks.rados.rados.0.smithi161.stdout:127: writing smithi161112115-127 from 1623437 to 2117682 tid 2 2023-01-24T19:16:30.935 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.935 INFO:tasks.rados.rados.0.smithi161.stdout:112: finishing write tid 3 to smithi161112115-112 2023-01-24T19:16:30.936 INFO:tasks.rados.rados.0.smithi161.stdout:112: finishing write tid 4 to smithi161112115-112 2023-01-24T19:16:30.936 INFO:tasks.rados.rados.0.smithi161.stdout:112: finishing write tid 5 to smithi161112115-112 2023-01-24T19:16:30.936 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 112 v 59 (ObjNum 111 snap 0 seq_num 111) dirty exists 2023-01-24T19:16:30.936 INFO:tasks.rados.rados.0.smithi161.stdout:112: left oid 112 (ObjNum 111 snap 0 seq_num 111) 2023-01-24T19:16:30.936 INFO:tasks.rados.rados.0.smithi161.stdout:113: finishing write tid 1 to smithi161112115-113 2023-01-24T19:16:30.937 INFO:tasks.rados.rados.0.smithi161.stdout:113: finishing write tid 2 to smithi161112115-113 2023-01-24T19:16:30.937 INFO:tasks.rados.rados.0.smithi161.stdout:112: done (15 left) 2023-01-24T19:16:30.937 INFO:tasks.rados.rados.0.smithi161.stdout:128: write initial oid 128 2023-01-24T19:16:30.937 INFO:tasks.rados.rados.0.smithi161.stdout:128: seq_num 127 ranges {534489=618271,1565566=568923} 2023-01-24T19:16:30.941 INFO:tasks.rados.rados.0.smithi161.stdout:128: writing smithi161112115-128 from 534489 to 1152760 tid 1 2023-01-24T19:16:30.947 INFO:tasks.rados.rados.0.smithi161.stdout:128: writing smithi161112115-128 from 1565566 to 2134489 tid 2 2023-01-24T19:16:30.947 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.948 INFO:tasks.rados.rados.0.smithi161.stdout:113: finishing write tid 3 to smithi161112115-113 2023-01-24T19:16:30.948 INFO:tasks.rados.rados.0.smithi161.stdout:113: finishing write tid 4 to smithi161112115-113 2023-01-24T19:16:30.948 INFO:tasks.rados.rados.0.smithi161.stdout:113: finishing write tid 5 to smithi161112115-113 2023-01-24T19:16:30.948 INFO:tasks.rados.rados.0.smithi161.stdout:113: finishing write tid 6 to smithi161112115-113 2023-01-24T19:16:30.948 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 113 v 64 (ObjNum 112 snap 0 seq_num 112) dirty exists 2023-01-24T19:16:30.949 INFO:tasks.rados.rados.0.smithi161.stdout:113: left oid 113 (ObjNum 112 snap 0 seq_num 112) 2023-01-24T19:16:30.949 INFO:tasks.rados.rados.0.smithi161.stdout:114: finishing write tid 1 to smithi161112115-114 2023-01-24T19:16:30.949 INFO:tasks.rados.rados.0.smithi161.stdout:113: done (15 left) 2023-01-24T19:16:30.949 INFO:tasks.rados.rados.0.smithi161.stdout:129: write initial oid 129 2023-01-24T19:16:30.949 INFO:tasks.rados.rados.0.smithi161.stdout:129: seq_num 128 ranges {551296=693520,2024048=127248} 2023-01-24T19:16:30.954 INFO:tasks.rados.rados.0.smithi161.stdout:129: writing smithi161112115-129 from 551296 to 1244816 tid 1 2023-01-24T19:16:30.959 INFO:tasks.rados.rados.0.smithi161.stdout:129: writing smithi161112115-129 from 2024048 to 2151296 tid 2 2023-01-24T19:16:30.959 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.960 INFO:tasks.rados.rados.0.smithi161.stdout:114: finishing write tid 2 to smithi161112115-114 2023-01-24T19:16:30.960 INFO:tasks.rados.rados.0.smithi161.stdout:114: finishing write tid 3 to smithi161112115-114 2023-01-24T19:16:30.960 INFO:tasks.rados.rados.0.smithi161.stdout:115: finishing write tid 1 to smithi161112115-115 2023-01-24T19:16:30.960 INFO:tasks.rados.rados.0.smithi161.stdout:115: finishing write tid 2 to smithi161112115-115 2023-01-24T19:16:30.960 INFO:tasks.rados.rados.0.smithi161.stdout:115: finishing write tid 3 to smithi161112115-115 2023-01-24T19:16:30.961 INFO:tasks.rados.rados.0.smithi161.stdout:115: finishing write tid 4 to smithi161112115-115 2023-01-24T19:16:30.961 INFO:tasks.rados.rados.0.smithi161.stdout:115: finishing write tid 5 to smithi161112115-115 2023-01-24T19:16:30.961 INFO:tasks.rados.rados.0.smithi161.stdout:115: finishing write tid 6 to smithi161112115-115 2023-01-24T19:16:30.961 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 115 v 64 (ObjNum 114 snap 0 seq_num 114) dirty exists 2023-01-24T19:16:30.961 INFO:tasks.rados.rados.0.smithi161.stdout:115: left oid 115 (ObjNum 114 snap 0 seq_num 114) 2023-01-24T19:16:30.962 INFO:tasks.rados.rados.0.smithi161.stdout:114: finishing write tid 4 to smithi161112115-114 2023-01-24T19:16:30.962 INFO:tasks.rados.rados.0.smithi161.stdout:114: finishing write tid 5 to smithi161112115-114 2023-01-24T19:16:30.962 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 114 v 57 (ObjNum 113 snap 0 seq_num 113) dirty exists 2023-01-24T19:16:30.962 INFO:tasks.rados.rados.0.smithi161.stdout:114: left oid 114 (ObjNum 113 snap 0 seq_num 113) 2023-01-24T19:16:30.962 INFO:tasks.rados.rados.0.smithi161.stdout:116: finishing write tid 1 to smithi161112115-116 2023-01-24T19:16:30.963 INFO:tasks.rados.rados.0.smithi161.stdout:116: finishing write tid 2 to smithi161112115-116 2023-01-24T19:16:30.963 INFO:tasks.rados.rados.0.smithi161.stdout:116: finishing write tid 3 to smithi161112115-116 2023-01-24T19:16:30.963 INFO:tasks.rados.rados.0.smithi161.stdout:116: finishing write tid 4 to smithi161112115-116 2023-01-24T19:16:30.963 INFO:tasks.rados.rados.0.smithi161.stdout:114: done (15 left) 2023-01-24T19:16:30.963 INFO:tasks.rados.rados.0.smithi161.stdout:115: done (14 left) 2023-01-24T19:16:30.964 INFO:tasks.rados.rados.0.smithi161.stdout:130: write initial oid 130 2023-01-24T19:16:30.964 INFO:tasks.rados.rados.0.smithi161.stdout:130: seq_num 129 ranges {568103=768769,2082530=85573} 2023-01-24T19:16:30.966 INFO:tasks.rados.rados.0.smithi161.stdout:130: writing smithi161112115-130 from 568103 to 1336872 tid 1 2023-01-24T19:16:30.971 INFO:tasks.rados.rados.0.smithi161.stdout:130: writing smithi161112115-130 from 2082530 to 2168103 tid 2 2023-01-24T19:16:30.972 INFO:tasks.rados.rados.0.smithi161.stdout:131: write initial oid 131 2023-01-24T19:16:30.972 INFO:tasks.rados.rados.0.smithi161.stdout:131: seq_num 130 ranges {584910=560371,1857365=327545} 2023-01-24T19:16:30.977 INFO:tasks.rados.rados.0.smithi161.stdout:131: writing smithi161112115-131 from 584910 to 1145281 tid 1 2023-01-24T19:16:30.984 INFO:tasks.rados.rados.0.smithi161.stdout:131: writing smithi161112115-131 from 1857365 to 2184910 tid 2 2023-01-24T19:16:30.984 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.984 INFO:tasks.rados.rados.0.smithi161.stdout:116: finishing write tid 5 to smithi161112115-116 2023-01-24T19:16:30.984 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 116 v 97 (ObjNum 115 snap 0 seq_num 115) dirty exists 2023-01-24T19:16:30.984 INFO:tasks.rados.rados.0.smithi161.stdout:116: left oid 116 (ObjNum 115 snap 0 seq_num 115) 2023-01-24T19:16:30.985 INFO:tasks.rados.rados.0.smithi161.stdout:117: finishing write tid 1 to smithi161112115-117 2023-01-24T19:16:30.985 INFO:tasks.rados.rados.0.smithi161.stdout:117: finishing write tid 2 to smithi161112115-117 2023-01-24T19:16:30.985 INFO:tasks.rados.rados.0.smithi161.stdout:116: done (15 left) 2023-01-24T19:16:30.985 INFO:tasks.rados.rados.0.smithi161.stdout:132: write initial oid 132 2023-01-24T19:16:30.985 INFO:tasks.rados.rados.0.smithi161.stdout:132: seq_num 131 ranges {601717=635620,1799494=402223} 2023-01-24T19:16:30.990 INFO:tasks.rados.rados.0.smithi161.stdout:132: writing smithi161112115-132 from 601717 to 1237337 tid 1 2023-01-24T19:16:30.996 INFO:tasks.rados.rados.0.smithi161.stdout:132: writing smithi161112115-132 from 1799494 to 2201717 tid 2 2023-01-24T19:16:30.996 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:30.996 INFO:tasks.rados.rados.0.smithi161.stdout:117: finishing write tid 3 to smithi161112115-117 2023-01-24T19:16:30.996 INFO:tasks.rados.rados.0.smithi161.stdout:117: finishing write tid 4 to smithi161112115-117 2023-01-24T19:16:30.996 INFO:tasks.rados.rados.0.smithi161.stdout:117: finishing write tid 5 to smithi161112115-117 2023-01-24T19:16:30.997 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 117 v 101 (ObjNum 116 snap 0 seq_num 116) dirty exists 2023-01-24T19:16:30.997 INFO:tasks.rados.rados.0.smithi161.stdout:117: left oid 117 (ObjNum 116 snap 0 seq_num 116) 2023-01-24T19:16:30.997 INFO:tasks.rados.rados.0.smithi161.stdout:118: finishing write tid 1 to smithi161112115-118 2023-01-24T19:16:30.997 INFO:tasks.rados.rados.0.smithi161.stdout:118: finishing write tid 2 to smithi161112115-118 2023-01-24T19:16:30.997 INFO:tasks.rados.rados.0.smithi161.stdout:118: finishing write tid 3 to smithi161112115-118 2023-01-24T19:16:30.998 INFO:tasks.rados.rados.0.smithi161.stdout:117: done (15 left) 2023-01-24T19:16:30.998 INFO:tasks.rados.rados.0.smithi161.stdout:133: write initial oid 133 2023-01-24T19:16:30.998 INFO:tasks.rados.rados.0.smithi161.stdout:133: seq_num 132 ranges {618524=710869,1857976=360548} 2023-01-24T19:16:31.002 INFO:tasks.rados.rados.0.smithi161.stdout:133: writing smithi161112115-133 from 618524 to 1329393 tid 1 2023-01-24T19:16:31.009 INFO:tasks.rados.rados.0.smithi161.stdout:133: writing smithi161112115-133 from 1857976 to 2218524 tid 2 2023-01-24T19:16:31.009 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.010 INFO:tasks.rados.rados.0.smithi161.stdout:118: finishing write tid 4 to smithi161112115-118 2023-01-24T19:16:31.010 INFO:tasks.rados.rados.0.smithi161.stdout:118: finishing write tid 5 to smithi161112115-118 2023-01-24T19:16:31.010 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 118 v 68 (ObjNum 117 snap 0 seq_num 117) dirty exists 2023-01-24T19:16:31.010 INFO:tasks.rados.rados.0.smithi161.stdout:118: left oid 118 (ObjNum 117 snap 0 seq_num 117) 2023-01-24T19:16:31.010 INFO:tasks.rados.rados.0.smithi161.stdout:119: finishing write tid 1 to smithi161112115-119 2023-01-24T19:16:31.011 INFO:tasks.rados.rados.0.smithi161.stdout:119: finishing write tid 2 to smithi161112115-119 2023-01-24T19:16:31.011 INFO:tasks.rados.rados.0.smithi161.stdout:119: finishing write tid 3 to smithi161112115-119 2023-01-24T19:16:31.011 INFO:tasks.rados.rados.0.smithi161.stdout:119: finishing write tid 4 to smithi161112115-119 2023-01-24T19:16:31.011 INFO:tasks.rados.rados.0.smithi161.stdout:119: finishing write tid 5 to smithi161112115-119 2023-01-24T19:16:31.011 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 119 v 105 (ObjNum 118 snap 0 seq_num 118) dirty exists 2023-01-24T19:16:31.011 INFO:tasks.rados.rados.0.smithi161.stdout:119: left oid 119 (ObjNum 118 snap 0 seq_num 118) 2023-01-24T19:16:31.012 INFO:tasks.rados.rados.0.smithi161.stdout:120: finishing write tid 1 to smithi161112115-120 2023-01-24T19:16:31.012 INFO:tasks.rados.rados.0.smithi161.stdout:120: finishing write tid 2 to smithi161112115-120 2023-01-24T19:16:31.012 INFO:tasks.rados.rados.0.smithi161.stdout:120: finishing write tid 3 to smithi161112115-120 2023-01-24T19:16:31.012 INFO:tasks.rados.rados.0.smithi161.stdout:120: finishing write tid 4 to smithi161112115-120 2023-01-24T19:16:31.012 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 120 v 59 (ObjNum 119 snap 0 seq_num 119) dirty exists 2023-01-24T19:16:31.013 INFO:tasks.rados.rados.0.smithi161.stdout:120: left oid 120 (ObjNum 119 snap 0 seq_num 119) 2023-01-24T19:16:31.013 INFO:tasks.rados.rados.0.smithi161.stdout:121: finishing write tid 1 to smithi161112115-121 2023-01-24T19:16:31.013 INFO:tasks.rados.rados.0.smithi161.stdout:121: finishing write tid 2 to smithi161112115-121 2023-01-24T19:16:31.013 INFO:tasks.rados.rados.0.smithi161.stdout:121: finishing write tid 3 to smithi161112115-121 2023-01-24T19:16:31.013 INFO:tasks.rados.rados.0.smithi161.stdout:121: finishing write tid 4 to smithi161112115-121 2023-01-24T19:16:31.014 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 121 v 71 (ObjNum 120 snap 0 seq_num 120) dirty exists 2023-01-24T19:16:31.014 INFO:tasks.rados.rados.0.smithi161.stdout:121: left oid 121 (ObjNum 120 snap 0 seq_num 120) 2023-01-24T19:16:31.014 INFO:tasks.rados.rados.0.smithi161.stdout:122: finishing write tid 1 to smithi161112115-122 2023-01-24T19:16:31.014 INFO:tasks.rados.rados.0.smithi161.stdout:122: finishing write tid 2 to smithi161112115-122 2023-01-24T19:16:31.014 INFO:tasks.rados.rados.0.smithi161.stdout:122: finishing write tid 3 to smithi161112115-122 2023-01-24T19:16:31.015 INFO:tasks.rados.rados.0.smithi161.stdout:122: finishing write tid 4 to smithi161112115-122 2023-01-24T19:16:31.015 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 122 v 74 (ObjNum 121 snap 0 seq_num 121) dirty exists 2023-01-24T19:16:31.015 INFO:tasks.rados.rados.0.smithi161.stdout:122: left oid 122 (ObjNum 121 snap 0 seq_num 121) 2023-01-24T19:16:31.015 INFO:tasks.rados.rados.0.smithi161.stdout:123: finishing write tid 1 to smithi161112115-123 2023-01-24T19:16:31.015 INFO:tasks.rados.rados.0.smithi161.stdout:123: finishing write tid 2 to smithi161112115-123 2023-01-24T19:16:31.016 INFO:tasks.rados.rados.0.smithi161.stdout:123: finishing write tid 3 to smithi161112115-123 2023-01-24T19:16:31.016 INFO:tasks.rados.rados.0.smithi161.stdout:123: finishing write tid 4 to smithi161112115-123 2023-01-24T19:16:31.016 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 123 v 77 (ObjNum 122 snap 0 seq_num 122) dirty exists 2023-01-24T19:16:31.016 INFO:tasks.rados.rados.0.smithi161.stdout:123: left oid 123 (ObjNum 122 snap 0 seq_num 122) 2023-01-24T19:16:31.016 INFO:tasks.rados.rados.0.smithi161.stdout:124: finishing write tid 1 to smithi161112115-124 2023-01-24T19:16:31.017 INFO:tasks.rados.rados.0.smithi161.stdout:124: finishing write tid 2 to smithi161112115-124 2023-01-24T19:16:31.017 INFO:tasks.rados.rados.0.smithi161.stdout:124: finishing write tid 3 to smithi161112115-124 2023-01-24T19:16:31.017 INFO:tasks.rados.rados.0.smithi161.stdout:118: done (15 left) 2023-01-24T19:16:31.017 INFO:tasks.rados.rados.0.smithi161.stdout:119: done (14 left) 2023-01-24T19:16:31.017 INFO:tasks.rados.rados.0.smithi161.stdout:120: done (13 left) 2023-01-24T19:16:31.018 INFO:tasks.rados.rados.0.smithi161.stdout:121: done (12 left) 2023-01-24T19:16:31.018 INFO:tasks.rados.rados.0.smithi161.stdout:122: done (11 left) 2023-01-24T19:16:31.018 INFO:tasks.rados.rados.0.smithi161.stdout:123: done (10 left) 2023-01-24T19:16:31.018 INFO:tasks.rados.rados.0.smithi161.stdout:134: write initial oid 134 2023-01-24T19:16:31.018 INFO:tasks.rados.rados.0.smithi161.stdout:134: seq_num 133 ranges {635331=786118,1916458=318873} 2023-01-24T19:16:31.019 INFO:tasks.rados.rados.0.smithi161.stdout:134: writing smithi161112115-134 from 635331 to 1421449 tid 1 2023-01-24T19:16:31.020 INFO:tasks.rados.rados.0.smithi161.stdout:134: writing smithi161112115-134 from 1916458 to 2235331 tid 2 2023-01-24T19:16:31.021 INFO:tasks.rados.rados.0.smithi161.stdout:135: write initial oid 135 2023-01-24T19:16:31.021 INFO:tasks.rados.rados.0.smithi161.stdout:135: seq_num 134 ranges {652138=461367,1574940=677198} 2023-01-24T19:16:31.026 INFO:tasks.rados.rados.0.smithi161.stdout:135: writing smithi161112115-135 from 652138 to 1113505 tid 1 2023-01-24T19:16:31.033 INFO:tasks.rados.rados.0.smithi161.stdout:135: writing smithi161112115-135 from 1574940 to 2252138 tid 2 2023-01-24T19:16:31.033 INFO:tasks.rados.rados.0.smithi161.stdout:136: write initial oid 136 2023-01-24T19:16:31.033 INFO:tasks.rados.rados.0.smithi161.stdout:136: seq_num 135 ranges {668945=536616,1633422=491363} 2023-01-24T19:16:31.040 INFO:tasks.rados.rados.0.smithi161.stdout:136: writing smithi161112115-136 from 668945 to 1205561 tid 1 2023-01-24T19:16:31.045 INFO:tasks.rados.rados.0.smithi161.stdout:136: writing smithi161112115-136 from 1633422 to 2124785 tid 2 2023-01-24T19:16:31.045 INFO:tasks.rados.rados.0.smithi161.stdout:137: write initial oid 137 2023-01-24T19:16:31.046 INFO:tasks.rados.rados.0.smithi161.stdout:137: seq_num 136 ranges {685752=611865,1975551=310201} 2023-01-24T19:16:31.052 INFO:tasks.rados.rados.0.smithi161.stdout:137: writing smithi161112115-137 from 685752 to 1297617 tid 1 2023-01-24T19:16:31.058 INFO:tasks.rados.rados.0.smithi161.stdout:137: writing smithi161112115-137 from 1975551 to 2285752 tid 2 2023-01-24T19:16:31.058 INFO:tasks.rados.rados.0.smithi161.stdout:138: write initial oid 138 2023-01-24T19:16:31.058 INFO:tasks.rados.rados.0.smithi161.stdout:138: seq_num 137 ranges {702559=403467,1750386=495032} 2023-01-24T19:16:31.064 INFO:tasks.rados.rados.0.smithi161.stdout:138: writing smithi161112115-138 from 702559 to 1106026 tid 1 2023-01-24T19:16:31.070 INFO:tasks.rados.rados.0.smithi161.stdout:138: writing smithi161112115-138 from 1750386 to 2245418 tid 2 2023-01-24T19:16:31.071 INFO:tasks.rados.rados.0.smithi161.stdout:139: write initial oid 139 2023-01-24T19:16:31.071 INFO:tasks.rados.rados.0.smithi161.stdout:139: seq_num 138 ranges {719366=478716,1808868=510498} 2023-01-24T19:16:31.077 INFO:tasks.rados.rados.0.smithi161.stdout:139: writing smithi161112115-139 from 719366 to 1198082 tid 1 2023-01-24T19:16:31.083 INFO:tasks.rados.rados.0.smithi161.stdout:139: writing smithi161112115-139 from 1808868 to 2319366 tid 2 2023-01-24T19:16:31.084 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.084 INFO:tasks.rados.rados.0.smithi161.stdout:124: finishing write tid 4 to smithi161112115-124 2023-01-24T19:16:31.084 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 124 v 67 (ObjNum 123 snap 0 seq_num 123) dirty exists 2023-01-24T19:16:31.084 INFO:tasks.rados.rados.0.smithi161.stdout:124: left oid 124 (ObjNum 123 snap 0 seq_num 123) 2023-01-24T19:16:31.084 INFO:tasks.rados.rados.0.smithi161.stdout:125: finishing write tid 1 to smithi161112115-125 2023-01-24T19:16:31.085 INFO:tasks.rados.rados.0.smithi161.stdout:125: finishing write tid 2 to smithi161112115-125 2023-01-24T19:16:31.085 INFO:tasks.rados.rados.0.smithi161.stdout:125: finishing write tid 3 to smithi161112115-125 2023-01-24T19:16:31.085 INFO:tasks.rados.rados.0.smithi161.stdout:124: done (15 left) 2023-01-24T19:16:31.085 INFO:tasks.rados.rados.0.smithi161.stdout:140: write initial oid 140 2023-01-24T19:16:31.086 INFO:tasks.rados.rados.0.smithi161.stdout:140: seq_num 139 ranges {736173=553965,1867350=468823} 2023-01-24T19:16:31.090 INFO:tasks.rados.rados.0.smithi161.stdout:140: writing smithi161112115-140 from 736173 to 1290138 tid 1 2023-01-24T19:16:31.096 INFO:tasks.rados.rados.0.smithi161.stdout:140: writing smithi161112115-140 from 1867350 to 2336173 tid 2 2023-01-24T19:16:31.096 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.097 INFO:tasks.rados.rados.0.smithi161.stdout:125: finishing write tid 4 to smithi161112115-125 2023-01-24T19:16:31.097 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 125 v 80 (ObjNum 124 snap 0 seq_num 124) dirty exists 2023-01-24T19:16:31.097 INFO:tasks.rados.rados.0.smithi161.stdout:125: left oid 125 (ObjNum 124 snap 0 seq_num 124) 2023-01-24T19:16:31.097 INFO:tasks.rados.rados.0.smithi161.stdout:126: finishing write tid 1 to smithi161112115-126 2023-01-24T19:16:31.097 INFO:tasks.rados.rados.0.smithi161.stdout:125: done (15 left) 2023-01-24T19:16:31.098 INFO:tasks.rados.rados.0.smithi161.stdout:141: write initial oid 141 2023-01-24T19:16:31.098 INFO:tasks.rados.rados.0.smithi161.stdout:141: seq_num 140 ranges {752980=629214,1809479=543501} 2023-01-24T19:16:31.104 INFO:tasks.rados.rados.0.smithi161.stdout:141: writing smithi161112115-141 from 752980 to 1382194 tid 1 2023-01-24T19:16:31.110 INFO:tasks.rados.rados.0.smithi161.stdout:141: writing smithi161112115-141 from 1809479 to 2352980 tid 2 2023-01-24T19:16:31.110 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.110 INFO:tasks.rados.rados.0.smithi161.stdout:126: finishing write tid 2 to smithi161112115-126 2023-01-24T19:16:31.110 INFO:tasks.rados.rados.0.smithi161.stdout:127: finishing write tid 1 to smithi161112115-127 2023-01-24T19:16:31.110 INFO:tasks.rados.rados.0.smithi161.stdout:126: finishing write tid 3 to smithi161112115-126 2023-01-24T19:16:31.111 INFO:tasks.rados.rados.0.smithi161.stdout:126: finishing write tid 4 to smithi161112115-126 2023-01-24T19:16:31.111 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 126 v 68 (ObjNum 125 snap 0 seq_num 125) dirty exists 2023-01-24T19:16:31.111 INFO:tasks.rados.rados.0.smithi161.stdout:126: left oid 126 (ObjNum 125 snap 0 seq_num 125) 2023-01-24T19:16:31.111 INFO:tasks.rados.rados.0.smithi161.stdout:127: finishing write tid 2 to smithi161112115-127 2023-01-24T19:16:31.111 INFO:tasks.rados.rados.0.smithi161.stdout:128: finishing write tid 1 to smithi161112115-128 2023-01-24T19:16:31.112 INFO:tasks.rados.rados.0.smithi161.stdout:127: finishing write tid 3 to smithi161112115-127 2023-01-24T19:16:31.112 INFO:tasks.rados.rados.0.smithi161.stdout:127: finishing write tid 4 to smithi161112115-127 2023-01-24T19:16:31.112 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 127 v 70 (ObjNum 126 snap 0 seq_num 126) dirty exists 2023-01-24T19:16:31.112 INFO:tasks.rados.rados.0.smithi161.stdout:127: left oid 127 (ObjNum 126 snap 0 seq_num 126) 2023-01-24T19:16:31.112 INFO:tasks.rados.rados.0.smithi161.stdout:128: finishing write tid 2 to smithi161112115-128 2023-01-24T19:16:31.113 INFO:tasks.rados.rados.0.smithi161.stdout:126: done (15 left) 2023-01-24T19:16:31.113 INFO:tasks.rados.rados.0.smithi161.stdout:127: done (14 left) 2023-01-24T19:16:31.113 INFO:tasks.rados.rados.0.smithi161.stdout:142: write initial oid 142 2023-01-24T19:16:31.113 INFO:tasks.rados.rados.0.smithi161.stdout:142: seq_num 141 ranges {769787=704463,2267961=101826} 2023-01-24T19:16:31.117 INFO:tasks.rados.rados.0.smithi161.stdout:142: writing smithi161112115-142 from 769787 to 1474250 tid 1 2023-01-24T19:16:31.122 INFO:tasks.rados.rados.0.smithi161.stdout:142: writing smithi161112115-142 from 2267961 to 2369787 tid 2 2023-01-24T19:16:31.123 INFO:tasks.rados.rados.0.smithi161.stdout:143: write initial oid 143 2023-01-24T19:16:31.123 INFO:tasks.rados.rados.0.smithi161.stdout:143: seq_num 142 ranges {786594=779712,2326443=60151} 2023-01-24T19:16:31.131 INFO:tasks.rados.rados.0.smithi161.stdout:143: writing smithi161112115-143 from 786594 to 1566306 tid 1 2023-01-24T19:16:31.136 INFO:tasks.rados.rados.0.smithi161.stdout:143: writing smithi161112115-143 from 2326443 to 2386594 tid 2 2023-01-24T19:16:31.136 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.137 INFO:tasks.rados.rados.0.smithi161.stdout:128: finishing write tid 3 to smithi161112115-128 2023-01-24T19:16:31.137 INFO:tasks.rados.rados.0.smithi161.stdout:128: finishing write tid 4 to smithi161112115-128 2023-01-24T19:16:31.137 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 128 v 72 (ObjNum 127 snap 0 seq_num 127) dirty exists 2023-01-24T19:16:31.137 INFO:tasks.rados.rados.0.smithi161.stdout:128: left oid 128 (ObjNum 127 snap 0 seq_num 127) 2023-01-24T19:16:31.137 INFO:tasks.rados.rados.0.smithi161.stdout:129: finishing write tid 1 to smithi161112115-129 2023-01-24T19:16:31.138 INFO:tasks.rados.rados.0.smithi161.stdout:128: done (15 left) 2023-01-24T19:16:31.138 INFO:tasks.rados.rados.0.smithi161.stdout:144: write initial oid 144 2023-01-24T19:16:31.138 INFO:tasks.rados.rados.0.smithi161.stdout:144: seq_num 143 ranges {403401=454961,1584925=506039} 2023-01-24T19:16:31.141 INFO:tasks.rados.rados.0.smithi161.stdout:144: writing smithi161112115-144 from 403401 to 858362 tid 1 2023-01-24T19:16:31.148 INFO:tasks.rados.rados.0.smithi161.stdout:144: writing smithi161112115-144 from 1584925 to 2090964 tid 2 2023-01-24T19:16:31.148 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.148 INFO:tasks.rados.rados.0.smithi161.stdout:129: finishing write tid 2 to smithi161112115-129 2023-01-24T19:16:31.149 INFO:tasks.rados.rados.0.smithi161.stdout:129: finishing write tid 3 to smithi161112115-129 2023-01-24T19:16:31.149 INFO:tasks.rados.rados.0.smithi161.stdout:129: finishing write tid 4 to smithi161112115-129 2023-01-24T19:16:31.149 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 129 v 108 (ObjNum 128 snap 0 seq_num 128) dirty exists 2023-01-24T19:16:31.149 INFO:tasks.rados.rados.0.smithi161.stdout:129: left oid 129 (ObjNum 128 snap 0 seq_num 128) 2023-01-24T19:16:31.149 INFO:tasks.rados.rados.0.smithi161.stdout:130: finishing write tid 1 to smithi161112115-130 2023-01-24T19:16:31.150 INFO:tasks.rados.rados.0.smithi161.stdout:130: finishing write tid 2 to smithi161112115-130 2023-01-24T19:16:31.150 INFO:tasks.rados.rados.0.smithi161.stdout:130: finishing write tid 3 to smithi161112115-130 2023-01-24T19:16:31.150 INFO:tasks.rados.rados.0.smithi161.stdout:129: done (15 left) 2023-01-24T19:16:31.150 INFO:tasks.rados.rados.0.smithi161.stdout:145: write initial oid 145 2023-01-24T19:16:31.150 INFO:tasks.rados.rados.0.smithi161.stdout:145: seq_num 144 ranges {420208=530210,1527054=766050} 2023-01-24T19:16:31.153 INFO:tasks.rados.rados.0.smithi161.stdout:145: writing smithi161112115-145 from 420208 to 950418 tid 1 2023-01-24T19:16:31.161 INFO:tasks.rados.rados.0.smithi161.stdout:145: writing smithi161112115-145 from 1527054 to 2293104 tid 2 2023-01-24T19:16:31.161 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.161 INFO:tasks.rados.rados.0.smithi161.stdout:130: finishing write tid 4 to smithi161112115-130 2023-01-24T19:16:31.162 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 130 v 111 (ObjNum 129 snap 0 seq_num 129) dirty exists 2023-01-24T19:16:31.162 INFO:tasks.rados.rados.0.smithi161.stdout:130: left oid 130 (ObjNum 129 snap 0 seq_num 129) 2023-01-24T19:16:31.162 INFO:tasks.rados.rados.0.smithi161.stdout:130: done (15 left) 2023-01-24T19:16:31.162 INFO:tasks.rados.rados.0.smithi161.stdout:146: write initial oid 146 2023-01-24T19:16:31.162 INFO:tasks.rados.rados.0.smithi161.stdout:146: seq_num 145 ranges {437015=721812,1701889=509708} 2023-01-24T19:16:31.167 INFO:tasks.rados.rados.0.smithi161.stdout:146: writing smithi161112115-146 from 437015 to 1158827 tid 1 2023-01-24T19:16:31.174 INFO:tasks.rados.rados.0.smithi161.stdout:146: writing smithi161112115-146 from 1701889 to 2211597 tid 2 2023-01-24T19:16:31.174 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.174 INFO:tasks.rados.rados.0.smithi161.stdout:131: finishing write tid 1 to smithi161112115-131 2023-01-24T19:16:31.174 INFO:tasks.rados.rados.0.smithi161.stdout:131: finishing write tid 2 to smithi161112115-131 2023-01-24T19:16:31.174 INFO:tasks.rados.rados.0.smithi161.stdout:131: finishing write tid 3 to smithi161112115-131 2023-01-24T19:16:31.175 INFO:tasks.rados.rados.0.smithi161.stdout:131: finishing write tid 4 to smithi161112115-131 2023-01-24T19:16:31.175 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 131 v 75 (ObjNum 130 snap 0 seq_num 130) dirty exists 2023-01-24T19:16:31.175 INFO:tasks.rados.rados.0.smithi161.stdout:131: left oid 131 (ObjNum 130 snap 0 seq_num 130) 2023-01-24T19:16:31.175 INFO:tasks.rados.rados.0.smithi161.stdout:132: finishing write tid 1 to smithi161112115-132 2023-01-24T19:16:31.175 INFO:tasks.rados.rados.0.smithi161.stdout:132: finishing write tid 2 to smithi161112115-132 2023-01-24T19:16:31.176 INFO:tasks.rados.rados.0.smithi161.stdout:131: done (15 left) 2023-01-24T19:16:31.176 INFO:tasks.rados.rados.0.smithi161.stdout:147: write initial oid 147 2023-01-24T19:16:31.176 INFO:tasks.rados.rados.0.smithi161.stdout:147: seq_num 146 ranges {453822=797061,1760371=653366} 2023-01-24T19:16:31.180 INFO:tasks.rados.rados.0.smithi161.stdout:147: writing smithi161112115-147 from 453822 to 1250883 tid 1 2023-01-24T19:16:31.187 INFO:tasks.rados.rados.0.smithi161.stdout:147: writing smithi161112115-147 from 1760371 to 2413737 tid 2 2023-01-24T19:16:31.187 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.187 INFO:tasks.rados.rados.0.smithi161.stdout:132: finishing write tid 3 to smithi161112115-132 2023-01-24T19:16:31.188 INFO:tasks.rados.rados.0.smithi161.stdout:132: finishing write tid 4 to smithi161112115-132 2023-01-24T19:16:31.188 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 132 v 61 (ObjNum 131 snap 0 seq_num 131) dirty exists 2023-01-24T19:16:31.188 INFO:tasks.rados.rados.0.smithi161.stdout:132: left oid 132 (ObjNum 131 snap 0 seq_num 131) 2023-01-24T19:16:31.188 INFO:tasks.rados.rados.0.smithi161.stdout:133: finishing write tid 1 to smithi161112115-133 2023-01-24T19:16:31.188 INFO:tasks.rados.rados.0.smithi161.stdout:133: finishing write tid 2 to smithi161112115-133 2023-01-24T19:16:31.189 INFO:tasks.rados.rados.0.smithi161.stdout:133: finishing write tid 3 to smithi161112115-133 2023-01-24T19:16:31.189 INFO:tasks.rados.rados.0.smithi161.stdout:133: finishing write tid 4 to smithi161112115-133 2023-01-24T19:16:31.189 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 133 v 78 (ObjNum 132 snap 0 seq_num 132) dirty exists 2023-01-24T19:16:31.189 INFO:tasks.rados.rados.0.smithi161.stdout:133: left oid 133 (ObjNum 132 snap 0 seq_num 132) 2023-01-24T19:16:31.189 INFO:tasks.rados.rados.0.smithi161.stdout:132: done (15 left) 2023-01-24T19:16:31.190 INFO:tasks.rados.rados.0.smithi161.stdout:133: done (14 left) 2023-01-24T19:16:31.190 INFO:tasks.rados.rados.0.smithi161.stdout:148: write initial oid 148 2023-01-24T19:16:31.190 INFO:tasks.rados.rados.0.smithi161.stdout:148: seq_num 147 ranges {470629=472310,1418853=513377} 2023-01-24T19:16:31.192 INFO:tasks.rados.rados.0.smithi161.stdout:148: writing smithi161112115-148 from 470629 to 942939 tid 1 2023-01-24T19:16:31.198 INFO:tasks.rados.rados.0.smithi161.stdout:148: writing smithi161112115-148 from 1418853 to 1932230 tid 2 2023-01-24T19:16:31.198 INFO:tasks.rados.rados.0.smithi161.stdout:149: write initial oid 149 2023-01-24T19:16:31.198 INFO:tasks.rados.rados.0.smithi161.stdout:149: seq_num 148 ranges {487436=547559,1760982=657035} 2023-01-24T19:16:31.204 INFO:tasks.rados.rados.0.smithi161.stdout:149: writing smithi161112115-149 from 487436 to 1034995 tid 1 2023-01-24T19:16:31.211 INFO:tasks.rados.rados.0.smithi161.stdout:149: writing smithi161112115-149 from 1760982 to 2418017 tid 2 2023-01-24T19:16:31.212 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.212 INFO:tasks.rados.rados.0.smithi161.stdout:134: finishing write tid 1 to smithi161112115-134 2023-01-24T19:16:31.212 INFO:tasks.rados.rados.0.smithi161.stdout:134: finishing write tid 2 to smithi161112115-134 2023-01-24T19:16:31.212 INFO:tasks.rados.rados.0.smithi161.stdout:134: finishing write tid 3 to smithi161112115-134 2023-01-24T19:16:31.213 INFO:tasks.rados.rados.0.smithi161.stdout:134: finishing write tid 4 to smithi161112115-134 2023-01-24T19:16:31.213 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 134 v 83 (ObjNum 133 snap 0 seq_num 133) dirty exists 2023-01-24T19:16:31.213 INFO:tasks.rados.rados.0.smithi161.stdout:134: left oid 134 (ObjNum 133 snap 0 seq_num 133) 2023-01-24T19:16:31.213 INFO:tasks.rados.rados.0.smithi161.stdout:135: finishing write tid 1 to smithi161112115-135 2023-01-24T19:16:31.213 INFO:tasks.rados.rados.0.smithi161.stdout:135: finishing write tid 2 to smithi161112115-135 2023-01-24T19:16:31.213 INFO:tasks.rados.rados.0.smithi161.stdout:135: finishing write tid 3 to smithi161112115-135 2023-01-24T19:16:31.214 INFO:tasks.rados.rados.0.smithi161.stdout:135: finishing write tid 4 to smithi161112115-135 2023-01-24T19:16:31.214 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 135 v 114 (ObjNum 134 snap 0 seq_num 134) dirty exists 2023-01-24T19:16:31.214 INFO:tasks.rados.rados.0.smithi161.stdout:135: left oid 135 (ObjNum 134 snap 0 seq_num 134) 2023-01-24T19:16:31.214 INFO:tasks.rados.rados.0.smithi161.stdout:136: finishing write tid 1 to smithi161112115-136 2023-01-24T19:16:31.214 INFO:tasks.rados.rados.0.smithi161.stdout:136: finishing write tid 2 to smithi161112115-136 2023-01-24T19:16:31.215 INFO:tasks.rados.rados.0.smithi161.stdout:134: done (15 left) 2023-01-24T19:16:31.215 INFO:tasks.rados.rados.0.smithi161.stdout:135: done (14 left) 2023-01-24T19:16:31.215 INFO:tasks.rados.rados.0.smithi161.stdout:150: write initial oid 150 2023-01-24T19:16:31.215 INFO:tasks.rados.rados.0.smithi161.stdout:150: seq_num 149 ranges {504243=622808,1819464=517046} 2023-01-24T19:16:31.217 INFO:tasks.rados.rados.0.smithi161.stdout:150: writing smithi161112115-150 from 504243 to 1127051 tid 1 2023-01-24T19:16:31.224 INFO:tasks.rados.rados.0.smithi161.stdout:150: writing smithi161112115-150 from 1819464 to 2336510 tid 2 2023-01-24T19:16:31.225 INFO:tasks.rados.rados.0.smithi161.stdout:151: write initial oid 151 2023-01-24T19:16:31.225 INFO:tasks.rados.rados.0.smithi161.stdout:151: seq_num 150 ranges {521050=698057,1877946=643104} 2023-01-24T19:16:31.231 INFO:tasks.rados.rados.0.smithi161.stdout:151: writing smithi161112115-151 from 521050 to 1219107 tid 1 2023-01-24T19:16:31.239 INFO:tasks.rados.rados.0.smithi161.stdout:151: writing smithi161112115-151 from 1877946 to 2521050 tid 2 2023-01-24T19:16:31.239 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.240 INFO:tasks.rados.rados.0.smithi161.stdout:136: finishing write tid 3 to smithi161112115-136 2023-01-24T19:16:31.240 INFO:tasks.rados.rados.0.smithi161.stdout:136: finishing write tid 4 to smithi161112115-136 2023-01-24T19:16:31.240 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 136 v 86 (ObjNum 135 snap 0 seq_num 135) dirty exists 2023-01-24T19:16:31.240 INFO:tasks.rados.rados.0.smithi161.stdout:136: left oid 136 (ObjNum 135 snap 0 seq_num 135) 2023-01-24T19:16:31.240 INFO:tasks.rados.rados.0.smithi161.stdout:137: finishing write tid 1 to smithi161112115-137 2023-01-24T19:16:31.241 INFO:tasks.rados.rados.0.smithi161.stdout:137: finishing write tid 2 to smithi161112115-137 2023-01-24T19:16:31.241 INFO:tasks.rados.rados.0.smithi161.stdout:136: done (15 left) 2023-01-24T19:16:31.241 INFO:tasks.rados.rados.0.smithi161.stdout:152: write initial oid 152 2023-01-24T19:16:31.241 INFO:tasks.rados.rados.0.smithi161.stdout:152: seq_num 151 ranges {537857=773306,1936428=520715} 2023-01-24T19:16:31.246 INFO:tasks.rados.rados.0.smithi161.stdout:152: writing smithi161112115-152 from 537857 to 1311163 tid 1 2023-01-24T19:16:31.253 INFO:tasks.rados.rados.0.smithi161.stdout:152: writing smithi161112115-152 from 1936428 to 2457143 tid 2 2023-01-24T19:16:31.253 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.253 INFO:tasks.rados.rados.0.smithi161.stdout:137: finishing write tid 3 to smithi161112115-137 2023-01-24T19:16:31.253 INFO:tasks.rados.rados.0.smithi161.stdout:137: finishing write tid 4 to smithi161112115-137 2023-01-24T19:16:31.253 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 137 v 61 (ObjNum 136 snap 0 seq_num 136) dirty exists 2023-01-24T19:16:31.254 INFO:tasks.rados.rados.0.smithi161.stdout:137: left oid 137 (ObjNum 136 snap 0 seq_num 136) 2023-01-24T19:16:31.254 INFO:tasks.rados.rados.0.smithi161.stdout:138: finishing write tid 1 to smithi161112115-138 2023-01-24T19:16:31.254 INFO:tasks.rados.rados.0.smithi161.stdout:138: finishing write tid 2 to smithi161112115-138 2023-01-24T19:16:31.254 INFO:tasks.rados.rados.0.smithi161.stdout:138: finishing write tid 3 to smithi161112115-138 2023-01-24T19:16:31.254 INFO:tasks.rados.rados.0.smithi161.stdout:138: finishing write tid 4 to smithi161112115-138 2023-01-24T19:16:31.255 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 138 v 89 (ObjNum 137 snap 0 seq_num 137) dirty exists 2023-01-24T19:16:31.255 INFO:tasks.rados.rados.0.smithi161.stdout:138: left oid 138 (ObjNum 137 snap 0 seq_num 137) 2023-01-24T19:16:31.255 INFO:tasks.rados.rados.0.smithi161.stdout:139: finishing write tid 1 to smithi161112115-139 2023-01-24T19:16:31.255 INFO:tasks.rados.rados.0.smithi161.stdout:137: done (15 left) 2023-01-24T19:16:31.255 INFO:tasks.rados.rados.0.smithi161.stdout:138: done (14 left) 2023-01-24T19:16:31.256 INFO:tasks.rados.rados.0.smithi161.stdout:153: write initial oid 153 2023-01-24T19:16:31.256 INFO:tasks.rados.rados.0.smithi161.stdout:153: seq_num 152 ranges {554664=448555,1478557=664373} 2023-01-24T19:16:31.258 INFO:tasks.rados.rados.0.smithi161.stdout:153: writing smithi161112115-153 from 554664 to 1003219 tid 1 2023-01-24T19:16:31.264 INFO:tasks.rados.rados.0.smithi161.stdout:153: writing smithi161112115-153 from 1478557 to 2142930 tid 2 2023-01-24T19:16:31.265 INFO:tasks.rados.rados.0.smithi161.stdout:154: write initial oid 154 2023-01-24T19:16:31.265 INFO:tasks.rados.rados.0.smithi161.stdout:154: seq_num 153 ranges {571471=640157,1653392=524384} 2023-01-24T19:16:31.271 INFO:tasks.rados.rados.0.smithi161.stdout:154: writing smithi161112115-154 from 571471 to 1211628 tid 1 2023-01-24T19:16:31.277 INFO:tasks.rados.rados.0.smithi161.stdout:154: writing smithi161112115-154 from 1653392 to 2177776 tid 2 2023-01-24T19:16:31.277 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.277 INFO:tasks.rados.rados.0.smithi161.stdout:139: finishing write tid 2 to smithi161112115-139 2023-01-24T19:16:31.278 INFO:tasks.rados.rados.0.smithi161.stdout:139: finishing write tid 3 to smithi161112115-139 2023-01-24T19:16:31.278 INFO:tasks.rados.rados.0.smithi161.stdout:139: finishing write tid 4 to smithi161112115-139 2023-01-24T19:16:31.278 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 139 v 92 (ObjNum 138 snap 0 seq_num 138) dirty exists 2023-01-24T19:16:31.278 INFO:tasks.rados.rados.0.smithi161.stdout:139: left oid 139 (ObjNum 138 snap 0 seq_num 138) 2023-01-24T19:16:31.278 INFO:tasks.rados.rados.0.smithi161.stdout:140: finishing write tid 1 to smithi161112115-140 2023-01-24T19:16:31.279 INFO:tasks.rados.rados.0.smithi161.stdout:140: finishing write tid 2 to smithi161112115-140 2023-01-24T19:16:31.279 INFO:tasks.rados.rados.0.smithi161.stdout:139: done (15 left) 2023-01-24T19:16:31.279 INFO:tasks.rados.rados.0.smithi161.stdout:155: write initial oid 155 2023-01-24T19:16:31.279 INFO:tasks.rados.rados.0.smithi161.stdout:155: seq_num 154 ranges {588278=715406,1711874=668042} 2023-01-24T19:16:31.284 INFO:tasks.rados.rados.0.smithi161.stdout:155: writing smithi161112115-155 from 588278 to 1303684 tid 1 2023-01-24T19:16:31.291 INFO:tasks.rados.rados.0.smithi161.stdout:155: writing smithi161112115-155 from 1711874 to 2379916 tid 2 2023-01-24T19:16:31.291 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.291 INFO:tasks.rados.rados.0.smithi161.stdout:140: finishing write tid 3 to smithi161112115-140 2023-01-24T19:16:31.291 INFO:tasks.rados.rados.0.smithi161.stdout:140: finishing write tid 4 to smithi161112115-140 2023-01-24T19:16:31.291 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 140 v 95 (ObjNum 139 snap 0 seq_num 139) dirty exists 2023-01-24T19:16:31.292 INFO:tasks.rados.rados.0.smithi161.stdout:140: left oid 140 (ObjNum 139 snap 0 seq_num 139) 2023-01-24T19:16:31.292 INFO:tasks.rados.rados.0.smithi161.stdout:141: finishing write tid 1 to smithi161112115-141 2023-01-24T19:16:31.292 INFO:tasks.rados.rados.0.smithi161.stdout:141: finishing write tid 2 to smithi161112115-141 2023-01-24T19:16:31.293 INFO:tasks.rados.rados.0.smithi161.stdout:141: finishing write tid 3 to smithi161112115-141 2023-01-24T19:16:31.293 INFO:tasks.rados.rados.0.smithi161.stdout:141: finishing write tid 4 to smithi161112115-141 2023-01-24T19:16:31.293 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 141 v 64 (ObjNum 140 snap 0 seq_num 140) dirty exists 2023-01-24T19:16:31.293 INFO:tasks.rados.rados.0.smithi161.stdout:141: left oid 141 (ObjNum 140 snap 0 seq_num 140) 2023-01-24T19:16:31.293 INFO:tasks.rados.rados.0.smithi161.stdout:142: finishing write tid 1 to smithi161112115-142 2023-01-24T19:16:31.294 INFO:tasks.rados.rados.0.smithi161.stdout:142: finishing write tid 2 to smithi161112115-142 2023-01-24T19:16:31.294 INFO:tasks.rados.rados.0.smithi161.stdout:142: finishing write tid 3 to smithi161112115-142 2023-01-24T19:16:31.294 INFO:tasks.rados.rados.0.smithi161.stdout:140: done (15 left) 2023-01-24T19:16:31.294 INFO:tasks.rados.rados.0.smithi161.stdout:141: done (14 left) 2023-01-24T19:16:31.294 INFO:tasks.rados.rados.0.smithi161.stdout:156: write initial oid 156 2023-01-24T19:16:31.294 INFO:tasks.rados.rados.0.smithi161.stdout:156: seq_num 155 ranges {605085=790655,2170356=434729} 2023-01-24T19:16:31.298 INFO:tasks.rados.rados.0.smithi161.stdout:156: writing smithi161112115-156 from 605085 to 1395740 tid 1 2023-01-24T19:16:31.304 INFO:tasks.rados.rados.0.smithi161.stdout:156: writing smithi161112115-156 from 2170356 to 2605085 tid 2 2023-01-24T19:16:31.305 INFO:tasks.rados.rados.0.smithi161.stdout:157: write initial oid 157 2023-01-24T19:16:31.305 INFO:tasks.rados.rados.0.smithi161.stdout:157: seq_num 156 ranges {621892=465904,1712485=671711} 2023-01-24T19:16:31.311 INFO:tasks.rados.rados.0.smithi161.stdout:157: writing smithi161112115-157 from 621892 to 1087796 tid 1 2023-01-24T19:16:31.318 INFO:tasks.rados.rados.0.smithi161.stdout:157: writing smithi161112115-157 from 1712485 to 2384196 tid 2 2023-01-24T19:16:31.319 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.319 INFO:tasks.rados.rados.0.smithi161.stdout:142: finishing write tid 4 to smithi161112115-142 2023-01-24T19:16:31.319 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 142 v 98 (ObjNum 141 snap 0 seq_num 141) dirty exists 2023-01-24T19:16:31.319 INFO:tasks.rados.rados.0.smithi161.stdout:142: left oid 142 (ObjNum 141 snap 0 seq_num 141) 2023-01-24T19:16:31.319 INFO:tasks.rados.rados.0.smithi161.stdout:143: finishing write tid 1 to smithi161112115-143 2023-01-24T19:16:31.320 INFO:tasks.rados.rados.0.smithi161.stdout:143: finishing write tid 2 to smithi161112115-143 2023-01-24T19:16:31.320 INFO:tasks.rados.rados.0.smithi161.stdout:142: done (15 left) 2023-01-24T19:16:31.320 INFO:tasks.rados.rados.0.smithi161.stdout:158: write initial oid 158 2023-01-24T19:16:31.320 INFO:tasks.rados.rados.0.smithi161.stdout:158: seq_num 157 ranges {638699=541153,1770967=531722} 2023-01-24T19:16:31.325 INFO:tasks.rados.rados.0.smithi161.stdout:158: writing smithi161112115-158 from 638699 to 1179852 tid 1 2023-01-24T19:16:31.331 INFO:tasks.rados.rados.0.smithi161.stdout:158: writing smithi161112115-158 from 1770967 to 2302689 tid 2 2023-01-24T19:16:31.331 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.332 INFO:tasks.rados.rados.0.smithi161.stdout:143: finishing write tid 3 to smithi161112115-143 2023-01-24T19:16:31.332 INFO:tasks.rados.rados.0.smithi161.stdout:143: finishing write tid 4 to smithi161112115-143 2023-01-24T19:16:31.332 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 143 v 101 (ObjNum 142 snap 0 seq_num 142) dirty exists 2023-01-24T19:16:31.332 INFO:tasks.rados.rados.0.smithi161.stdout:143: left oid 143 (ObjNum 142 snap 0 seq_num 142) 2023-01-24T19:16:31.332 INFO:tasks.rados.rados.0.smithi161.stdout:144: finishing write tid 1 to smithi161112115-144 2023-01-24T19:16:31.333 INFO:tasks.rados.rados.0.smithi161.stdout:144: finishing write tid 2 to smithi161112115-144 2023-01-24T19:16:31.333 INFO:tasks.rados.rados.0.smithi161.stdout:144: finishing write tid 3 to smithi161112115-144 2023-01-24T19:16:31.333 INFO:tasks.rados.rados.0.smithi161.stdout:143: done (15 left) 2023-01-24T19:16:31.333 INFO:tasks.rados.rados.0.smithi161.stdout:159: write initial oid 159 2023-01-24T19:16:31.334 INFO:tasks.rados.rados.0.smithi161.stdout:159: seq_num 158 ranges {655506=616402,1829449=675380} 2023-01-24T19:16:31.338 INFO:tasks.rados.rados.0.smithi161.stdout:159: writing smithi161112115-159 from 655506 to 1271908 tid 1 2023-01-24T19:16:31.345 INFO:tasks.rados.rados.0.smithi161.stdout:159: writing smithi161112115-159 from 1829449 to 2504829 tid 2 2023-01-24T19:16:31.346 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.346 INFO:tasks.rados.rados.0.smithi161.stdout:144: finishing write tid 4 to smithi161112115-144 2023-01-24T19:16:31.346 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 144 v 117 (ObjNum 143 snap 0 seq_num 143) dirty exists 2023-01-24T19:16:31.346 INFO:tasks.rados.rados.0.smithi161.stdout:144: left oid 144 (ObjNum 143 snap 0 seq_num 143) 2023-01-24T19:16:31.347 INFO:tasks.rados.rados.0.smithi161.stdout:145: finishing write tid 1 to smithi161112115-145 2023-01-24T19:16:31.347 INFO:tasks.rados.rados.0.smithi161.stdout:145: finishing write tid 2 to smithi161112115-145 2023-01-24T19:16:31.347 INFO:tasks.rados.rados.0.smithi161.stdout:145: finishing write tid 3 to smithi161112115-145 2023-01-24T19:16:31.347 INFO:tasks.rados.rados.0.smithi161.stdout:145: finishing write tid 4 to smithi161112115-145 2023-01-24T19:16:31.347 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 145 v 104 (ObjNum 144 snap 0 seq_num 144) dirty exists 2023-01-24T19:16:31.347 INFO:tasks.rados.rados.0.smithi161.stdout:145: left oid 145 (ObjNum 144 snap 0 seq_num 144) 2023-01-24T19:16:31.348 INFO:tasks.rados.rados.0.smithi161.stdout:146: finishing write tid 1 to smithi161112115-146 2023-01-24T19:16:31.348 INFO:tasks.rados.rados.0.smithi161.stdout:146: finishing write tid 2 to smithi161112115-146 2023-01-24T19:16:31.348 INFO:tasks.rados.rados.0.smithi161.stdout:146: finishing write tid 3 to smithi161112115-146 2023-01-24T19:16:31.348 INFO:tasks.rados.rados.0.smithi161.stdout:146: finishing write tid 4 to smithi161112115-146 2023-01-24T19:16:31.349 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 146 v 67 (ObjNum 145 snap 0 seq_num 145) dirty exists 2023-01-24T19:16:31.349 INFO:tasks.rados.rados.0.smithi161.stdout:146: left oid 146 (ObjNum 145 snap 0 seq_num 145) 2023-01-24T19:16:31.349 INFO:tasks.rados.rados.0.smithi161.stdout:147: finishing write tid 1 to smithi161112115-147 2023-01-24T19:16:31.349 INFO:tasks.rados.rados.0.smithi161.stdout:147: finishing write tid 2 to smithi161112115-147 2023-01-24T19:16:31.349 INFO:tasks.rados.rados.0.smithi161.stdout:147: finishing write tid 3 to smithi161112115-147 2023-01-24T19:16:31.350 INFO:tasks.rados.rados.0.smithi161.stdout:147: finishing write tid 4 to smithi161112115-147 2023-01-24T19:16:31.350 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 147 v 81 (ObjNum 146 snap 0 seq_num 146) dirty exists 2023-01-24T19:16:31.350 INFO:tasks.rados.rados.0.smithi161.stdout:147: left oid 147 (ObjNum 146 snap 0 seq_num 146) 2023-01-24T19:16:31.350 INFO:tasks.rados.rados.0.smithi161.stdout:148: finishing write tid 1 to smithi161112115-148 2023-01-24T19:16:31.350 INFO:tasks.rados.rados.0.smithi161.stdout:148: finishing write tid 2 to smithi161112115-148 2023-01-24T19:16:31.351 INFO:tasks.rados.rados.0.smithi161.stdout:148: finishing write tid 3 to smithi161112115-148 2023-01-24T19:16:31.351 INFO:tasks.rados.rados.0.smithi161.stdout:148: finishing write tid 4 to smithi161112115-148 2023-01-24T19:16:31.351 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 148 v 84 (ObjNum 147 snap 0 seq_num 147) dirty exists 2023-01-24T19:16:31.351 INFO:tasks.rados.rados.0.smithi161.stdout:148: left oid 148 (ObjNum 147 snap 0 seq_num 147) 2023-01-24T19:16:31.351 INFO:tasks.rados.rados.0.smithi161.stdout:149: finishing write tid 1 to smithi161112115-149 2023-01-24T19:16:31.352 INFO:tasks.rados.rados.0.smithi161.stdout:149: finishing write tid 2 to smithi161112115-149 2023-01-24T19:16:31.352 INFO:tasks.rados.rados.0.smithi161.stdout:149: finishing write tid 3 to smithi161112115-149 2023-01-24T19:16:31.352 INFO:tasks.rados.rados.0.smithi161.stdout:149: finishing write tid 4 to smithi161112115-149 2023-01-24T19:16:31.352 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 149 v 120 (ObjNum 148 snap 0 seq_num 148) dirty exists 2023-01-24T19:16:31.352 INFO:tasks.rados.rados.0.smithi161.stdout:149: left oid 149 (ObjNum 148 snap 0 seq_num 148) 2023-01-24T19:16:31.353 INFO:tasks.rados.rados.0.smithi161.stdout:150: finishing write tid 1 to smithi161112115-150 2023-01-24T19:16:31.353 INFO:tasks.rados.rados.0.smithi161.stdout:150: finishing write tid 2 to smithi161112115-150 2023-01-24T19:16:31.353 INFO:tasks.rados.rados.0.smithi161.stdout:150: finishing write tid 3 to smithi161112115-150 2023-01-24T19:16:31.353 INFO:tasks.rados.rados.0.smithi161.stdout:150: finishing write tid 4 to smithi161112115-150 2023-01-24T19:16:31.353 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 150 v 107 (ObjNum 149 snap 0 seq_num 149) dirty exists 2023-01-24T19:16:31.354 INFO:tasks.rados.rados.0.smithi161.stdout:150: left oid 150 (ObjNum 149 snap 0 seq_num 149) 2023-01-24T19:16:31.354 INFO:tasks.rados.rados.0.smithi161.stdout:151: finishing write tid 1 to smithi161112115-151 2023-01-24T19:16:31.354 INFO:tasks.rados.rados.0.smithi161.stdout:151: finishing write tid 2 to smithi161112115-151 2023-01-24T19:16:31.354 INFO:tasks.rados.rados.0.smithi161.stdout:151: finishing write tid 3 to smithi161112115-151 2023-01-24T19:16:31.354 INFO:tasks.rados.rados.0.smithi161.stdout:151: finishing write tid 4 to smithi161112115-151 2023-01-24T19:16:31.355 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 151 v 70 (ObjNum 150 snap 0 seq_num 150) dirty exists 2023-01-24T19:16:31.355 INFO:tasks.rados.rados.0.smithi161.stdout:151: left oid 151 (ObjNum 150 snap 0 seq_num 150) 2023-01-24T19:16:31.355 INFO:tasks.rados.rados.0.smithi161.stdout:152: finishing write tid 1 to smithi161112115-152 2023-01-24T19:16:31.355 INFO:tasks.rados.rados.0.smithi161.stdout:152: finishing write tid 2 to smithi161112115-152 2023-01-24T19:16:31.355 INFO:tasks.rados.rados.0.smithi161.stdout:152: finishing write tid 3 to smithi161112115-152 2023-01-24T19:16:31.356 INFO:tasks.rados.rados.0.smithi161.stdout:144: done (15 left) 2023-01-24T19:16:31.356 INFO:tasks.rados.rados.0.smithi161.stdout:145: done (14 left) 2023-01-24T19:16:31.356 INFO:tasks.rados.rados.0.smithi161.stdout:146: done (13 left) 2023-01-24T19:16:31.356 INFO:tasks.rados.rados.0.smithi161.stdout:147: done (12 left) 2023-01-24T19:16:31.356 INFO:tasks.rados.rados.0.smithi161.stdout:148: done (11 left) 2023-01-24T19:16:31.357 INFO:tasks.rados.rados.0.smithi161.stdout:149: done (10 left) 2023-01-24T19:16:31.357 INFO:tasks.rados.rados.0.smithi161.stdout:150: done (9 left) 2023-01-24T19:16:31.357 INFO:tasks.rados.rados.0.smithi161.stdout:151: done (8 left) 2023-01-24T19:16:31.357 INFO:tasks.rados.rados.0.smithi161.stdout:160: write initial oid 160 2023-01-24T19:16:31.357 INFO:tasks.rados.rados.0.smithi161.stdout:160: seq_num 159 ranges {672313=691651,1887931=419038} 2023-01-24T19:16:31.358 INFO:tasks.rados.rados.0.smithi161.stdout:160: writing smithi161112115-160 from 672313 to 1363964 tid 1 2023-01-24T19:16:31.359 INFO:tasks.rados.rados.0.smithi161.stdout:160: writing smithi161112115-160 from 1887931 to 2306969 tid 2 2023-01-24T19:16:31.359 INFO:tasks.rados.rados.0.smithi161.stdout:161: write initial oid 161 2023-01-24T19:16:31.359 INFO:tasks.rados.rados.0.smithi161.stdout:161: seq_num 160 ranges {689120=483253,1946413=679049} 2023-01-24T19:16:31.365 INFO:tasks.rados.rados.0.smithi161.stdout:161: writing smithi161112115-161 from 689120 to 1172373 tid 1 2023-01-24T19:16:31.373 INFO:tasks.rados.rados.0.smithi161.stdout:161: writing smithi161112115-161 from 1946413 to 2625462 tid 2 2023-01-24T19:16:31.373 INFO:tasks.rados.rados.0.smithi161.stdout:162: write initial oid 162 2023-01-24T19:16:31.373 INFO:tasks.rados.rados.0.smithi161.stdout:162: seq_num 161 ranges {705927=558502,2004895=422707} 2023-01-24T19:16:31.380 INFO:tasks.rados.rados.0.smithi161.stdout:162: writing smithi161112115-162 from 705927 to 1264429 tid 1 2023-01-24T19:16:31.386 INFO:tasks.rados.rados.0.smithi161.stdout:162: writing smithi161112115-162 from 2004895 to 2427602 tid 2 2023-01-24T19:16:31.386 INFO:tasks.rados.rados.0.smithi161.stdout:163: write initial oid 163 2023-01-24T19:16:31.387 INFO:tasks.rados.rados.0.smithi161.stdout:163: seq_num 162 ranges {722734=633751,2063377=659357} 2023-01-24T19:16:31.394 INFO:tasks.rados.rados.0.smithi161.stdout:163: writing smithi161112115-163 from 722734 to 1356485 tid 1 2023-01-24T19:16:31.401 INFO:tasks.rados.rados.0.smithi161.stdout:163: writing smithi161112115-163 from 2063377 to 2722734 tid 2 2023-01-24T19:16:31.402 INFO:tasks.rados.rados.0.smithi161.stdout:164: write initial oid 164 2023-01-24T19:16:31.402 INFO:tasks.rados.rados.0.smithi161.stdout:164: seq_num 163 ranges {739541=709000,2121859=426376} 2023-01-24T19:16:31.409 INFO:tasks.rados.rados.0.smithi161.stdout:164: writing smithi161112115-164 from 739541 to 1448541 tid 1 2023-01-24T19:16:31.416 INFO:tasks.rados.rados.0.smithi161.stdout:164: writing smithi161112115-164 from 2121859 to 2548235 tid 2 2023-01-24T19:16:31.416 INFO:tasks.rados.rados.0.smithi161.stdout:165: write initial oid 165 2023-01-24T19:16:31.416 INFO:tasks.rados.rados.0.smithi161.stdout:165: seq_num 164 ranges {756348=784249,2063988=686387} 2023-01-24T19:16:31.424 INFO:tasks.rados.rados.0.smithi161.stdout:165: writing smithi161112115-165 from 756348 to 1540597 tid 1 2023-01-24T19:16:31.431 INFO:tasks.rados.rados.0.smithi161.stdout:165: writing smithi161112115-165 from 2063988 to 2750375 tid 2 2023-01-24T19:16:31.432 INFO:tasks.rados.rados.0.smithi161.stdout:166: write initial oid 166 2023-01-24T19:16:31.432 INFO:tasks.rados.rados.0.smithi161.stdout:166: seq_num 165 ranges {773155=459498,1722470=430045,2648676=124479} 2023-01-24T19:16:31.438 INFO:tasks.rados.rados.0.smithi161.stdout:166: writing smithi161112115-166 from 773155 to 1232653 tid 1 2023-01-24T19:16:31.444 INFO:tasks.rados.rados.0.smithi161.stdout:166: writing smithi161112115-166 from 1722470 to 2152515 tid 2 2023-01-24T19:16:31.447 INFO:tasks.rados.rados.0.smithi161.stdout:166: writing smithi161112115-166 from 2648676 to 2773155 tid 3 2023-01-24T19:16:31.447 INFO:tasks.rados.rados.0.smithi161.stdout:167: write initial oid 167 2023-01-24T19:16:31.447 INFO:tasks.rados.rados.0.smithi161.stdout:167: seq_num 166 ranges {789962=534747,1780952=690056} 2023-01-24T19:16:31.454 INFO:tasks.rados.rados.0.smithi161.stdout:167: writing smithi161112115-167 from 789962 to 1324709 tid 1 2023-01-24T19:16:31.462 INFO:tasks.rados.rados.0.smithi161.stdout:167: writing smithi161112115-167 from 1780952 to 2471008 tid 2 2023-01-24T19:16:31.462 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.463 INFO:tasks.rados.rados.0.smithi161.stdout:152: finishing write tid 4 to smithi161112115-152 2023-01-24T19:16:31.463 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 152 v 87 (ObjNum 151 snap 0 seq_num 151) dirty exists 2023-01-24T19:16:31.463 INFO:tasks.rados.rados.0.smithi161.stdout:152: left oid 152 (ObjNum 151 snap 0 seq_num 151) 2023-01-24T19:16:31.463 INFO:tasks.rados.rados.0.smithi161.stdout:153: finishing write tid 1 to smithi161112115-153 2023-01-24T19:16:31.463 INFO:tasks.rados.rados.0.smithi161.stdout:153: finishing write tid 2 to smithi161112115-153 2023-01-24T19:16:31.464 INFO:tasks.rados.rados.0.smithi161.stdout:153: finishing write tid 3 to smithi161112115-153 2023-01-24T19:16:31.464 INFO:tasks.rados.rados.0.smithi161.stdout:153: finishing write tid 4 to smithi161112115-153 2023-01-24T19:16:31.464 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 153 v 90 (ObjNum 152 snap 0 seq_num 152) dirty exists 2023-01-24T19:16:31.464 INFO:tasks.rados.rados.0.smithi161.stdout:153: left oid 153 (ObjNum 152 snap 0 seq_num 152) 2023-01-24T19:16:31.465 INFO:tasks.rados.rados.0.smithi161.stdout:154: finishing write tid 1 to smithi161112115-154 2023-01-24T19:16:31.465 INFO:tasks.rados.rados.0.smithi161.stdout:154: finishing write tid 2 to smithi161112115-154 2023-01-24T19:16:31.465 INFO:tasks.rados.rados.0.smithi161.stdout:152: done (15 left) 2023-01-24T19:16:31.465 INFO:tasks.rados.rados.0.smithi161.stdout:153: done (14 left) 2023-01-24T19:16:31.465 INFO:tasks.rados.rados.0.smithi161.stdout:168: write initial oid 168 2023-01-24T19:16:31.466 INFO:tasks.rados.rados.0.smithi161.stdout:154: finishing write tid 3 to smithi161112115-154 2023-01-24T19:16:31.466 INFO:tasks.rados.rados.0.smithi161.stdout:154: finishing write tid 4 to smithi161112115-154 2023-01-24T19:16:31.466 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 154 v 110 (ObjNum 153 snap 0 seq_num 153) dirty exists 2023-01-24T19:16:31.466 INFO:tasks.rados.rados.0.smithi161.stdout:154: left oid 154 (ObjNum 153 snap 0 seq_num 153) 2023-01-24T19:16:31.466 INFO:tasks.rados.rados.0.smithi161.stdout:155: finishing write tid 1 to smithi161112115-155 2023-01-24T19:16:31.467 INFO:tasks.rados.rados.0.smithi161.stdout:155: finishing write tid 2 to smithi161112115-155 2023-01-24T19:16:31.467 INFO:tasks.rados.rados.0.smithi161.stdout:155: finishing write tid 3 to smithi161112115-155 2023-01-24T19:16:31.467 INFO:tasks.rados.rados.0.smithi161.stdout:155: finishing write tid 4 to smithi161112115-155 2023-01-24T19:16:31.467 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 155 v 73 (ObjNum 154 snap 0 seq_num 154) dirty exists 2023-01-24T19:16:31.467 INFO:tasks.rados.rados.0.smithi161.stdout:155: left oid 155 (ObjNum 154 snap 0 seq_num 154) 2023-01-24T19:16:31.468 INFO:tasks.rados.rados.0.smithi161.stdout:156: finishing write tid 1 to smithi161112115-156 2023-01-24T19:16:31.468 INFO:tasks.rados.rados.0.smithi161.stdout:156: finishing write tid 2 to smithi161112115-156 2023-01-24T19:16:31.468 INFO:tasks.rados.rados.0.smithi161.stdout:156: finishing write tid 3 to smithi161112115-156 2023-01-24T19:16:31.468 INFO:tasks.rados.rados.0.smithi161.stdout:156: finishing write tid 4 to smithi161112115-156 2023-01-24T19:16:31.468 INFO:tasks.rados.rados.0.smithi161.stdout:update_object_version oid 156 v 93 (ObjNum 155 snap 0 seq_num 155) dirty exists 2023-01-24T19:16:31.469 INFO:tasks.rados.rados.0.smithi161.stdout:156: left oid 156 (ObjNum 155 snap 0 seq_num 155) 2023-01-24T19:16:31.469 INFO:tasks.rados.rados.0.smithi161.stdout:157: finishing write tid 1 to smithi161112115-157 2023-01-24T19:16:31.469 INFO:tasks.rados.rados.0.smithi161.stdout:168: seq_num 167 ranges {406769=609996,1439434=433714,2303522=503247} 2023-01-24T19:16:31.469 INFO:tasks.rados.rados.0.smithi161.stdout:168: writing smithi161112115-168 from 406769 to 1016765 tid 1 2023-01-24T19:16:31.472 INFO:tasks.rados.rados.0.smithi161.stdout:168: writing smithi161112115-168 from 1439434 to 1873148 tid 2 2023-01-24T19:16:31.477 INFO:tasks.rados.rados.0.smithi161.stdout:168: writing smithi161112115-168 from 2303522 to 2806769 tid 3 2023-01-24T19:16:31.477 INFO:tasks.rados.rados.0.smithi161.stdout:154: done (14 left) 2023-01-24T19:16:31.478 INFO:tasks.rados.rados.0.smithi161.stdout:155: done (13 left) 2023-01-24T19:16:31.478 INFO:tasks.rados.rados.0.smithi161.stdout:156: done (12 left) 2023-01-24T19:16:31.478 INFO:tasks.rados.rados.0.smithi161.stdout:169: write initial oid 169 2023-01-24T19:16:31.478 INFO:tasks.rados.rados.0.smithi161.stdout:169: seq_num 168 ranges {423576=401598,1497916=693725} 2023-01-24T19:16:31.482 INFO:tasks.rados.rados.0.smithi161.stdout:169: writing smithi161112115-169 from 423576 to 825174 tid 1 2023-01-24T19:16:31.489 INFO:tasks.rados.rados.0.smithi161.stdout:169: writing smithi161112115-169 from 1497916 to 2191641 tid 2 2023-01-24T19:16:31.490 INFO:tasks.rados.rados.0.smithi161.stdout:170: write initial oid 170 2023-01-24T19:16:31.490 INFO:tasks.rados.rados.0.smithi161.stdout:170: seq_num 169 ranges {440383=476847,1556398=437383,2474721=365662} 2023-01-24T19:16:31.495 INFO:tasks.rados.rados.0.smithi161.stdout:170: writing smithi161112115-170 from 440383 to 917230 tid 1 2023-01-24T19:16:31.501 INFO:tasks.rados.rados.0.smithi161.stdout:170: writing smithi161112115-170 from 1556398 to 1993781 tid 2 2023-01-24T19:16:31.506 INFO:tasks.rados.rados.0.smithi161.stdout:170: writing smithi161112115-170 from 2474721 to 2840383 tid 3 2023-01-24T19:16:31.506 INFO:tasks.rados.rados.0.smithi161.stdout:171: write initial oid 171 2023-01-24T19:16:31.506 INFO:tasks.rados.rados.0.smithi161.stdout:171: seq_num 170 ranges {457190=552096,1614880=581041,2785791=71399} 2023-01-24T19:16:31.511 INFO:tasks.rados.rados.0.smithi161.stdout:171: writing smithi161112115-171 from 457190 to 1009286 tid 1 2023-01-24T19:16:31.518 INFO:tasks.rados.rados.0.smithi161.stdout:171: writing smithi161112115-171 from 1614880 to 2195921 tid 2 2023-01-24T19:16:31.522 INFO:tasks.rados.rados.0.smithi161.stdout:171: writing smithi161112115-171 from 2785791 to 2857190 tid 3 2023-01-24T19:16:31.522 INFO:tasks.rados.rados.0.smithi161.stdout:172: write initial oid 172 2023-01-24T19:16:31.522 INFO:tasks.rados.rados.0.smithi161.stdout:172: seq_num 171 ranges {473997=627345,1673362=441052,2529567=344430} 2023-01-24T19:16:31.528 INFO:tasks.rados.rados.0.smithi161.stdout:172: writing smithi161112115-172 from 473997 to 1101342 tid 1 2023-01-24T19:16:31.534 INFO:tasks.rados.rados.0.smithi161.stdout:172: writing smithi161112115-172 from 1673362 to 2114414 tid 2 2023-01-24T19:16:31.538 INFO:tasks.rados.rados.0.smithi161.stdout:172: writing smithi161112115-172 from 2529567 to 2873997 tid 3 2023-01-24T19:16:31.539 INFO:tasks.rados.rados.0.smithi161.stdout: waiting on 16 2023-01-24T19:16:31.539 INFO:tasks.rados.rados.0.smithi161.stdout:157: finishing write tid 2 to smithi161112115-157 2023-01-24T19:16:31.539 INFO:tasks.rados.rados.0.smithi161.stdout:157: finishing write tid 3 to smithi161112115-157 2023-01-24T19:16:31.539 INFO:tasks.rados.rados.0.smithi161.stdout:157: finishing write tid 4 to smithi161112115-157 2023-01-24T19:16:31.539 INFO:tasks.rados.rados