2023-01-24T14:16:28.961 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T14:16:28.962 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} --name rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi --archive /home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311 --owner scheduled_rzarzynski@teuthology -- /home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311/orig.config.yaml 2023-01-24T14:16:28.990 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T14:16:29.061 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311 branch: main description: crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} email: null first_in_suite: false job_id: '7135311' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: main ceph: conf: global: ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(CACHE_POOL_NO_HIT_SET\) - \(POOL_APP_NOT_ENABLED\) - is full \(reached quota - \(POOL_FULL\) sha1: 1a671c198559f54389cd5afa22085a0c7b29a1ef ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: osd default pool size: 2 install: ceph: flavor: crimson sha1: 1a671c198559f54389cd5afa22085a0c7b29a1ef selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 510284b66513490445619d1430aa869868c71a09 owner: scheduled_rzarzynski@teuthology priority: 60 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 sha1: 1a671c198559f54389cd5afa22085a0c7b29a1ef sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 510284b66513490445619d1430aa869868c71a09 targets: smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjmRpV2nHf7zNb3iCW6xG4he1DmKPZ5qBdD8qr1YwZ1S7uwiCJ5kV1hlufB/jJbinbEvAiq3NmQjHfUZpWVJvEvQUguaoqr2pag64PI5VS3FSCB4nFiCXD5zqYASyxVG1EcUSBqJBi8Hh3DQoumOVbT6J1di3pqgo13JsATpZLBd342M1EoYyW1ia/0D5llnKznvJq996rtGm2RXZ50u+1VxpteD3Pf/tZuIe2hgJThto6ZZsoCBtNf0MuIWY7hqJpltje6lhzMCkmK9zYkccrYrRtDj2LORcXaWrv6Hw6a0sR8fAHQ/v7Tb9lyTH6wFAcCRom2xN9GuWhqyVJkJRaudordVVaj2tTKFZIQ2/ax4K59QnSK9KCyS/+nBg+tVxQTdBYIKA5rNii8xymJZ8iB+nBlpmBVgTA/ilanqp95QtsMYb/Na68E0nnm/1o5nkb9E3/oQPtN8xPGmVvmomyvtN667o9Nnzas3jR0CV/L1OGwmF0Ba96de4Bti0Qu8k= tasks: - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 flavor: crimson - workunit: clients: client.0: - rbd/test_librbd.sh teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_14:08:39 tube: smithi user: rzarzynski verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T14:16:29.062 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa; will attempt to use it 2023-01-24T14:16:29.062 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks 2023-01-24T14:16:29.063 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T14:16:29.064 INFO:teuthology.task.internal:Checking packages... 2023-01-24T14:16:29.092 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'crimson' and ceph hash '1a671c198559f54389cd5afa22085a0c7b29a1ef' 2023-01-24T14:16:29.092 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T14:16:29.093 INFO:teuthology.packaging:ref: None 2023-01-24T14:16:29.093 INFO:teuthology.packaging:tag: None 2023-01-24T14:16:29.093 INFO:teuthology.packaging:branch: main 2023-01-24T14:16:29.094 INFO:teuthology.packaging:sha1: 1a671c198559f54389cd5afa22085a0c7b29a1ef 2023-01-24T14:16:29.094 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&ref=main 2023-01-24T14:16:29.308 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1954.g1a671c19 2023-01-24T14:16:29.308 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T14:16:29.318 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T14:16:29.318 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T14:16:29.325 INFO:teuthology.task.internal:Saving configuration 2023-01-24T14:16:29.336 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T14:16:29.342 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T14:16:29.373 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 14:11:25.142536', 'locked_by': 'scheduled_rzarzynski@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBF/z5TPec7GLJk59JfqA3UqS05sBHKZTphtCTYRKZgBhOIQq+yyfwAPbtJZ7OWioSeyEFomyGbsxGFRKSI+1e1g='} 2023-01-24T14:16:29.373 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T14:16:29.380 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2023-01-24T14:16:29.380 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T14:16:29.420 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3e0d6d45e0>, signals=[15]) 2023-01-24T14:16:29.421 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T14:16:29.428 INFO:teuthology.task.internal:Opening connections... 2023-01-24T14:16:29.428 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T14:16:29.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T14:16:29.496 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T14:16:29.503 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2023-01-24T14:16:29.522 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2023-01-24T14:16:29.522 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2023-01-24T14:16:29.580 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2023-01-24T14:16:29.580 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8" 2023-01-24T14:16:29.580 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2023-01-24T14:16:29.581 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2023-01-24T14:16:29.581 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8" 2023-01-24T14:16:29.581 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2023-01-24T14:16:29.581 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-24T14:16:29.581 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2023-01-24T14:16:29.582 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-24T14:16:29.582 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2023-01-24T14:16:29.582 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T14:16:29.582 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T14:16:29.582 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-24T14:16:29.583 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2023-01-24T14:16:29.602 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T14:16:29.616 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T14:16:29.624 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T14:16:29.625 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2023-01-24T14:16:29.643 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T14:16:29.654 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T14:16:29.654 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2023-01-24T14:16:29.717 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T14:16:29.755 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T14:16:29.770 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T14:16:29.771 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T14:16:29.771 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T14:16:29.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T14:16:29.790 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2023-01-24T14:16:29.848 INFO:teuthology.orchestra.run.smithi105.stdout:4.18.0-408.el8.x86_64 2023-01-24T14:16:29.848 INFO:teuthology.task.kernel:Running kernel on smithi105: 4.18.0-408.el8.x86_64 2023-01-24T14:16:29.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2023-01-24T14:16:33.478 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - AppStream 38 MB/s | 27 MB 00:00 2023-01-24T14:16:39.011 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - BaseOS 40 MB/s | 27 MB 00:00 2023-01-24T14:16:43.294 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - Extras 126 kB/s | 18 kB 00:00 2023-01-24T14:16:43.796 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - PowerTools 15 MB/s | 5.6 MB 00:00 2023-01-24T14:16:45.004 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-01-24T14:16:46.112 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-24T14:16:49.429 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 476 kB/s | 24 kB 00:00 2023-01-24T14:16:52.578 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-24T14:16:52.578 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-24T14:16:52.734 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:16:52.735 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:16:52.735 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:16:52.735 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:16:52.736 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:16:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-24T14:16:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-24T14:16:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-24T14:16:52.736 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:16:52.737 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:16:52.737 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:16:52.737 INFO:teuthology.orchestra.run.smithi105.stdout:Install 3 Packages 2023-01-24T14:16:52.737 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:16:52.738 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 85 M 2023-01-24T14:16:52.738 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 94 M 2023-01-24T14:16:52.738 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:16:53.248 INFO:teuthology.orchestra.run.smithi105.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 18 MB/s | 9.2 MB 00:00 2023-01-24T14:16:54.006 INFO:teuthology.orchestra.run.smithi105.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 33 MB/s | 41 MB 00:01 2023-01-24T14:16:54.414 INFO:teuthology.orchestra.run.smithi105.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 20 MB/s | 34 MB 00:01 2023-01-24T14:16:54.417 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:16:54.417 INFO:teuthology.orchestra.run.smithi105.stdout:Total 50 MB/s | 85 MB 00:01 2023-01-24T14:16:55.054 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:16:55.453 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:16:55.453 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:16:58.396 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:16:58.419 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:17:01.113 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:17:01.786 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-24T14:17:05.666 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-24T14:17:06.091 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-24T14:17:09.408 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-24T14:17:09.691 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-24T14:17:31.453 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-24T14:17:31.478 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-24T14:17:31.915 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-24T14:17:31.916 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-24T14:17:31.916 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-24T14:17:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-24T14:17:32.488 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:17:32.489 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:17:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-24T14:17:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-24T14:17:32.489 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:17:32.490 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:17:32.604 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel --last | head -n 1 2023-01-24T14:17:32.806 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-4.18.0-448.el8.x86_64 Tue 24 Jan 2023 02:17:09 PM UTC 2023-01-24T14:17:32.808 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-24T14:17:32.808 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-24T14:17:32.809 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-24T14:17:32.809 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-24T14:17:32.809 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-01-24T14:17:32.809 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2023-01-24T14:17:32.809 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel --last | head -n 1 2023-01-24T14:17:32.995 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-4.18.0-448.el8.x86_64 Tue 24 Jan 2023 02:17:09 PM UTC 2023-01-24T14:17:32.997 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-24T14:17:32.998 DEBUG:teuthology.orchestra.run.smithi105:> sudo rpm -qi grub2-tools 2023-01-24T14:17:33.031 INFO:teuthology.orchestra.run.smithi105.stdout:Name : grub2-tools 2023-01-24T14:17:33.031 INFO:teuthology.orchestra.run.smithi105.stdout:Epoch : 1 2023-01-24T14:17:33.031 INFO:teuthology.orchestra.run.smithi105.stdout:Version : 2.02 2023-01-24T14:17:33.031 INFO:teuthology.orchestra.run.smithi105.stdout:Release : 129.el8 2023-01-24T14:17:33.031 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: x86_64 2023-01-24T14:17:33.032 INFO:teuthology.orchestra.run.smithi105.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-24T14:17:33.032 INFO:teuthology.orchestra.run.smithi105.stdout:Group : System Environment/Base 2023-01-24T14:17:33.032 INFO:teuthology.orchestra.run.smithi105.stdout:Size : 9265586 2023-01-24T14:17:33.032 INFO:teuthology.orchestra.run.smithi105.stdout:License : GPLv3+ 2023-01-24T14:17:33.033 INFO:teuthology.orchestra.run.smithi105.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-24T14:17:33.033 INFO:teuthology.orchestra.run.smithi105.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-24T14:17:33.033 INFO:teuthology.orchestra.run.smithi105.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-24T14:17:33.033 INFO:teuthology.orchestra.run.smithi105.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-24T14:17:33.033 INFO:teuthology.orchestra.run.smithi105.stdout:Relocations : (not relocatable) 2023-01-24T14:17:33.034 INFO:teuthology.orchestra.run.smithi105.stdout:Packager : CentOS BuildSystem 2023-01-24T14:17:33.034 INFO:teuthology.orchestra.run.smithi105.stdout:Vendor : CentOS 2023-01-24T14:17:33.034 INFO:teuthology.orchestra.run.smithi105.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-24T14:17:33.034 INFO:teuthology.orchestra.run.smithi105.stdout:Summary : Support tools for GRUB. 2023-01-24T14:17:33.034 INFO:teuthology.orchestra.run.smithi105.stdout:Description : 2023-01-24T14:17:33.035 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:17:33.035 INFO:teuthology.orchestra.run.smithi105.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-24T14:17:33.035 INFO:teuthology.orchestra.run.smithi105.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-24T14:17:33.035 INFO:teuthology.orchestra.run.smithi105.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-24T14:17:33.035 INFO:teuthology.orchestra.run.smithi105.stdout:hardware devices. 2023-01-24T14:17:33.035 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:17:33.036 INFO:teuthology.orchestra.run.smithi105.stdout:This subpackage provides tools for support of all platforms. 2023-01-24T14:17:33.037 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-24T14:17:33.037 INFO:teuthology.task.kernel:Updating grub on smithi105 to boot 4.18.0-448.el8.x86_64 2023-01-24T14:17:33.037 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-24T14:17:34.933 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2023-01-24T14:17:37.196 INFO:teuthology.orchestra.run.smithi105.stderr:done 2023-01-24T14:17:37.197 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2023-01-24T14:17:37.228 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.yNHDZKyQ2n 2023-01-24T14:17:37.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yNHDZKyQ2n 2023-01-24T14:17:37.326 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.yNHDZKyQ2n 2023-01-24T14:17:37.442 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.yNHDZKyQ2n is 6KB 2023-01-24T14:17:37.493 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.yNHDZKyQ2n 2023-01-24T14:17:37.510 DEBUG:teuthology.orchestra.run.smithi105:> sudo /bin/ls /boot/loader/entries || true 2023-01-24T14:17:37.587 INFO:teuthology.orchestra.run.smithi105.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-24T14:17:37.587 INFO:teuthology.orchestra.run.smithi105.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-24T14:17:37.587 INFO:teuthology.orchestra.run.smithi105.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-24T14:17:37.587 INFO:teuthology.orchestra.run.smithi105.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-24T14:17:37.588 DEBUG:teuthology.orchestra.run.smithi105:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-24T14:17:37.747 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2023-01-24T14:18:07.780 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-01-24T14:18:07.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-01-24T14:18:07.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T14:18:42.408 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.105 2023-01-24T14:19:12.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-01-24T14:19:12.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T14:19:15.755 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-01-24T14:19:16.067 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-01-24T14:19:16.067 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-448.el8.x86_64"... 2023-01-24T14:19:16.068 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2023-01-24T14:19:16.086 INFO:teuthology.orchestra.run.smithi105.stdout:4.18.0-448.el8.x86_64 2023-01-24T14:19:16.086 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-24T14:19:16.086 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-24T14:19:16.087 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-24T14:19:17.088 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T14:19:17.088 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T14:19:17.192 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2023-01-24T14:19:17.219 DEBUG:teuthology.parallel:result is None 2023-01-24T14:19:17.219 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T14:19:17.226 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T14:19:17.227 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T14:19:17.246 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T14:19:17.254 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T14:19:17.263 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T14:19:17.264 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T14:19:17.322 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T14:19:17.329 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T14:19:17.329 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T14:19:17.402 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T14:19:17.413 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T14:19:17.414 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T14:19:17.422 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T14:19:17.423 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T14:19:17.483 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T14:19:17.491 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T14:19:17.492 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T14:19:17.541 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T14:19:17.622 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T14:19:17.669 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T14:19:17.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T14:19:17.797 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:19:17.797 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T14:19:17.874 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2023-01-24T14:19:17.978 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T14:19:19.753 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T14:19:19.762 INFO:teuthology.task.internal:Starting timer... 2023-01-24T14:19:19.762 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T14:19:19.771 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T14:19:19.780 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T14:19:19.781 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2023-01-24T14:19:19.867 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2023-01-24T14:19:19.933 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T14:19:19.934 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2023-01-24T14:19:19.954 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2023-01-24T14:19:19.955 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive'} 2023-01-24T14:19:19.955 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T14:19:20.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T14:19:20.022 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T14:19:20.023 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2023-01-24T14:19:20.108 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T14:19:20.137 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-24T14:19:20.142 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-24T14:19:20.262 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T14:19:20.280 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T14:19:20.281 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T14:25:41.623 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com')] 2023-01-24T14:25:41.625 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2023-01-24T14:25:41.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T14:25:41.698 DEBUG:teuthology.orchestra.run.smithi105:> true 2023-01-24T14:25:41.724 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T14:25:41.734 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T14:25:41.734 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T14:25:41.735 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T14:25:41.796 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T14:25:41.810 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T14:25:41.872 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2023-01-24T14:25:41.883 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2023-01-24T14:25:41.897 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T14:25:41.911 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T14:25:41.975 INFO:teuthology.orchestra.run.smithi105.stderr:bash: ntpq: command not found 2023-01-24T14:25:41.979 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T14:25:41.979 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2023-01-24T14:25:41.979 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T14:25:41.979 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T14:25:41.979 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T14:25:41.980 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T14:25:41.981 INFO:teuthology.run_tasks:Running task install... 2023-01-24T14:25:41.995 DEBUG:teuthology.task.install:project ceph 2023-01-24T14:25:41.996 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'crimson', 'sha1': '1a671c198559f54389cd5afa22085a0c7b29a1ef'}} 2023-01-24T14:25:41.996 DEBUG:teuthology.task.install:config {'flavor': 'crimson', 'sha1': '1a671c198559f54389cd5afa22085a0c7b29a1ef'} 2023-01-24T14:25:41.996 INFO:teuthology.task.install:Using flavor: crimson 2023-01-24T14:25:42.003 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T14:25:42.003 INFO:teuthology.task.install:extra packages: [] 2023-01-24T14:25:42.004 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2023-01-24T14:25:42.058 INFO:teuthology.orchestra.run.smithi105.stdout:CentOSStream 2023-01-24T14:25:42.059 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T14:25:42.060 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'crimson', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1a671c198559f54389cd5afa22085a0c7b29a1ef', 'tag': None, 'wait_for_package': False} 2023-01-24T14:25:42.060 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2023-01-24T14:25:42.135 INFO:teuthology.orchestra.run.smithi105.stdout:CentOSStream 2023-01-24T14:25:42.137 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T14:25:42.137 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=1a671c198559f54389cd5afa22085a0c7b29a1ef 2023-01-24T14:25:42.300 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/main/1a671c198559f54389cd5afa22085a0c7b29a1ef/centos/8/flavors/crimson/ 2023-01-24T14:25:42.301 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1954.g1a671c19 2023-01-24T14:25:42.472 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/main/1a671c198559f54389cd5afa22085a0c7b29a1ef/centos/8/flavors/crimson/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/main/1a671c198559f54389cd5afa22085a0c7b29a1ef/centos/8/flavors/crimson/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/main/1a671c198559f54389cd5afa22085a0c7b29a1ef/centos/8/flavors/crimson/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T14:25:42.472 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:25:42.473 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T14:25:42.501 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T14:25:42.501 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1a671c198559f54389cd5afa22085a0c7b29a1ef/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T14:25:42.570 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T14:25:42.648 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T14:25:42.715 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2023-01-24T14:25:42.716 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2023-01-24T14:25:42.964 INFO:teuthology.orchestra.run.smithi105.stdout:61 files removed 2023-01-24T14:25:42.992 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw 2023-01-24T14:25:43.828 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 192 kB/s | 83 kB 00:00 2023-01-24T14:25:44.112 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 38 kB/s | 9.3 kB 00:00 2023-01-24T14:25:44.454 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 5.0 kB/s | 1.6 kB 00:00 2023-01-24T14:25:45.140 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - AppStream 42 MB/s | 27 MB 00:00 2023-01-24T14:25:50.733 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - BaseOS 43 MB/s | 27 MB 00:00 2023-01-24T14:25:54.990 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - Extras 146 kB/s | 18 kB 00:00 2023-01-24T14:25:55.434 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - PowerTools 18 MB/s | 5.6 MB 00:00 2023-01-24T14:25:56.650 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-24T14:25:58.259 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 9.0 MB/s | 13 MB 00:01 2023-01-24T14:26:01.509 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 511 kB/s | 24 kB 00:00 2023-01-24T14:26:05.025 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:26:05.027 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:26:05.028 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:26:05.028 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:26:05.028 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:26:05.028 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 14 M 2023-01-24T14:26:05.028 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2023-01-24T14:26:05.029 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 3.3 M 2023-01-24T14:26:05.029 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 3.5 M 2023-01-24T14:26:05.029 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:26:05.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 6.3 M 2023-01-24T14:26:05.029 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 27 M 2023-01-24T14:26:05.030 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 24 k 2023-01-24T14:26:05.030 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T14:26:05.030 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 741 k 2023-01-24T14:26:05.030 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T14:26:05.030 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-24T14:26:05.031 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 461 k 2023-01-24T14:26:05.031 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-24T14:26:05.031 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 7.2 M 2023-01-24T14:26:05.031 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T14:26:05.031 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-24T14:26:05.032 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse 2023-01-24T14:26:05.032 INFO:teuthology.orchestra.run.smithi105.stdout: x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 45 k 2023-01-24T14:26:05.032 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 115 k 2023-01-24T14:26:05.032 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 190 k 2023-01-24T14:26:05.032 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-24T14:26:05.033 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 371 k 2023-01-24T14:26:05.033 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 364 k 2023-01-24T14:26:05.033 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 106 k 2023-01-24T14:26:05.033 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T14:26:05.033 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:26:05.034 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:26:05.034 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:26:05.034 INFO:teuthology.orchestra.run.smithi105.stdout:Install 21 Packages 2023-01-24T14:26:05.034 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 2 Packages 2023-01-24T14:26:05.034 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:26:05.035 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 67 M 2023-01-24T14:26:05.035 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:26:06.604 INFO:teuthology.orchestra.run.smithi105.stdout:(1/23): ceph-base-18.0.0-1954.g1a671c19.el8.x86 4.0 MB/s | 6.3 MB 00:01 2023-01-24T14:26:06.677 INFO:teuthology.orchestra.run.smithi105.stdout:(2/23): ceph-selinux-18.0.0-1954.g1a671c19.el8. 335 kB/s | 24 kB 00:00 2023-01-24T14:26:06.869 INFO:teuthology.orchestra.run.smithi105.stdout:(3/23): libcephfs2-18.0.0-1954.g1a671c19.el8.x8 3.8 MB/s | 741 kB 00:00 2023-01-24T14:26:07.094 INFO:teuthology.orchestra.run.smithi105.stdout:(4/23): libradosstriper1-18.0.0-1954.g1a671c19. 2.0 MB/s | 461 kB 00:00 2023-01-24T14:26:07.486 INFO:teuthology.orchestra.run.smithi105.stdout:(5/23): ceph-radosgw-18.0.0-1954.g1a671c19.el8. 5.8 MB/s | 14 MB 00:02 2023-01-24T14:26:07.603 INFO:teuthology.orchestra.run.smithi105.stdout:(6/23): python3-ceph-argparse-18.0.0-1954.g1a67 383 kB/s | 45 kB 00:00 2023-01-24T14:26:07.937 INFO:teuthology.orchestra.run.smithi105.stdout:(7/23): ceph-common-18.0.0-1954.g1a671c19.el8.x 9.3 MB/s | 27 MB 00:02 2023-01-24T14:26:07.954 INFO:teuthology.orchestra.run.smithi105.stdout:(8/23): python3-ceph-common-18.0.0-1954.g1a671c 328 kB/s | 115 kB 00:00 2023-01-24T14:26:08.046 INFO:teuthology.orchestra.run.smithi105.stdout:(9/23): librgw2-18.0.0-1954.g1a671c19.el8.x86_6 7.5 MB/s | 7.2 MB 00:00 2023-01-24T14:26:08.063 INFO:teuthology.orchestra.run.smithi105.stdout:(10/23): python3-cephfs-18.0.0-1954.g1a671c19.e 1.5 MB/s | 190 kB 00:00 2023-01-24T14:26:08.113 INFO:teuthology.orchestra.run.smithi105.stdout:(11/23): python3-rados-18.0.0-1954.g1a671c19.el 2.3 MB/s | 371 kB 00:00 2023-01-24T14:26:08.138 INFO:teuthology.orchestra.run.smithi105.stdout:(12/23): python3-rbd-18.0.0-1954.g1a671c19.el8. 3.9 MB/s | 364 kB 00:00 2023-01-24T14:26:08.172 INFO:teuthology.orchestra.run.smithi105.stdout:(13/23): python3-rgw-18.0.0-1954.g1a671c19.el8. 978 kB/s | 106 kB 00:00 2023-01-24T14:26:08.297 INFO:teuthology.orchestra.run.smithi105.stdout:(14/23): librdkafka-0.11.4-3.el8.x86_64.rpm 1.9 MB/s | 354 kB 00:00 2023-01-24T14:26:08.347 INFO:teuthology.orchestra.run.smithi105.stdout:(15/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.2 MB/s | 259 kB 00:00 2023-01-24T14:26:08.373 INFO:teuthology.orchestra.run.smithi105.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 226 kB/s | 44 kB 00:00 2023-01-24T14:26:08.398 INFO:teuthology.orchestra.run.smithi105.stdout:(17/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 465 kB/s | 47 kB 00:00 2023-01-24T14:26:08.515 INFO:teuthology.orchestra.run.smithi105.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 414 kB/s | 59 kB 00:00 2023-01-24T14:26:08.540 INFO:teuthology.orchestra.run.smithi105.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 528 kB/s | 75 kB 00:00 2023-01-24T14:26:08.590 INFO:teuthology.orchestra.run.smithi105.stdout:(20/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2023-01-24T14:26:08.799 INFO:teuthology.orchestra.run.smithi105.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 6.1 MB/s | 1.7 MB 00:00 2023-01-24T14:26:08.866 INFO:teuthology.orchestra.run.smithi105.stdout:(22/23): librados2-18.0.0-1954.g1a671c19.el8.x8 10 MB/s | 3.3 MB 00:00 2023-01-24T14:26:08.925 INFO:teuthology.orchestra.run.smithi105.stdout:(23/23): librbd1-18.0.0-1954.g1a671c19.el8.x86_ 11 MB/s | 3.5 MB 00:00 2023-01-24T14:26:08.928 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:26:08.928 INFO:teuthology.orchestra.run.smithi105.stdout:Total 17 MB/s | 67 MB 00:03 2023-01-24T14:26:08.967 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:26:09.012 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:26:09.012 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:26:09.648 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:26:09.652 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:26:10.850 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:26:11.072 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/25 2023-01-24T14:26:11.268 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-24T14:26:11.664 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-24T14:26:11.870 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:18.0.0-1954.g1a671c19.el8.x86_64 3/25 2023-01-24T14:26:11.965 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:18.0.0-1954.g1a671c19.el8.x86_64 3/25 2023-01-24T14:26:12.246 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:18.0.0-1954.g1a671c19.el8.x86_64 4/25 2023-01-24T14:26:12.420 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:18.0.0-1954.g1a671c19.el8.x86_64 5/25 2023-01-24T14:26:12.841 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:18.0.0-1954.g1a671c19.el8.x86_64 5/25 2023-01-24T14:26:13.045 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:18.0.0-1954.g1a671c19.el8.x86_64 6/25 2023-01-24T14:26:13.254 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:18.0.0-1954.g1a671c19.el8.x86_64 6/25 2023-01-24T14:26:13.450 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.13.0-2.el8.x86_64 7/25 2023-01-24T14:26:13.617 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T14:26:13.770 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-24T14:26:13.850 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-24T14:26:13.980 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:18.0.0-1954.g1a671c19.el8. 10/25 2023-01-24T14:26:14.196 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:18.0.0-1954.g1a671c19.el8.x86_64 11/25 2023-01-24T14:26:14.446 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:18.0.0-1954.g1a671c19.el8.x86_64 12/25 2023-01-24T14:26:14.820 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:18.0.0-1954.g1a671c19.el8.x86_6 13/25 2023-01-24T14:26:15.056 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1954.g1a671c19.el8.x86_6 13/25 2023-01-24T14:26:15.257 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-24T14:26:16.181 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-24T14:26:16.411 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:18.0.0-1954.g1a671c19.el8.x86_64 16/25 2023-01-24T14:26:16.486 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:18.0.0-1954.g1a671c19.el8.x86_64 16/25 2023-01-24T14:26:16.711 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:18.0.0-1954.g1a671c19.el8.x86_64 17/25 2023-01-24T14:26:16.919 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-24T14:26:17.094 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:18.0.0-1954.g1a671c19.el8.x8 19/25 2023-01-24T14:26:21.392 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.0.0-1954.g1a671c19.el8.x86_64 20/25 2023-01-24T14:26:21.635 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:18.0.0-1954.g1a671c19.el8.x86_64 20/25 2023-01-24T14:26:22.368 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.0.0-1954.g1a671c19.el8.x86_64 20/25 2023-01-24T14:26:22.722 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:18.0.0-1954.g1a671c19.el8.x86_64 21/25 2023-01-24T14:26:22.805 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:18.0.0-1954.g1a671c19.el8.x86_64 21/25 2023-01-24T14:26:22.968 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:18.0.0-1954.g1a671c19.el8.x86_64 22/25 2023-01-24T14:26:42.672 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1954.g1a671c19.el8.x86_64 22/25 2023-01-24T14:26:42.673 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2023-01-24T14:26:42.673 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2023-01-24T14:26:42.674 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2023-01-24T14:26:42.674 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /run 2023-01-24T14:26:42.674 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2023-01-24T14:26:42.675 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2023-01-24T14:26:42.675 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2023-01-24T14:26:42.676 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2023-01-24T14:26:42.676 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2023-01-24T14:26:42.676 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:26:44.302 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:18.0.0-1954.g1a671c19.el8.x86_64 23/25 2023-01-24T14:26:44.323 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1954.g1a671c19.el8.x86_64 23/25 2023-01-24T14:26:44.323 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T14:26:44.560 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T14:26:44.560 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T14:26:45.281 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T14:26:45.281 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:18.0.0-1954.g1a671c19.el8.x86_64 1/25 2023-01-24T14:26:45.281 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:18.0.0-1954.g1a671c19.el8.x86_64 2/25 2023-01-24T14:26:45.282 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:18.0.0-1954.g1a671c19.el8.x86_64 3/25 2023-01-24T14:26:45.282 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:18.0.0-1954.g1a671c19.el8.x86_64 4/25 2023-01-24T14:26:45.282 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:18.0.0-1954.g1a671c19.el8.x86_64 5/25 2023-01-24T14:26:45.282 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:18.0.0-1954.g1a671c19.el8.x86_6 6/25 2023-01-24T14:26:45.282 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:18.0.0-1954.g1a671c19.el8.x86_64 7/25 2023-01-24T14:26:45.283 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1954.g1a671c19.el8. 8/25 2023-01-24T14:26:45.283 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:18.0.0-1954.g1a671c19.el8.x8 9/25 2023-01-24T14:26:45.283 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:18.0.0-1954.g1a671c19.el8.x86_64 10/25 2023-01-24T14:26:45.283 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:18.0.0-1954.g1a671c19.el8.x86_64 11/25 2023-01-24T14:26:45.283 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:18.0.0-1954.g1a671c19.el8.x86_64 12/25 2023-01-24T14:26:45.284 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:18.0.0-1954.g1a671c19.el8.x86_64 13/25 2023-01-24T14:26:45.284 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-24T14:26:45.284 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-24T14:26:45.284 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T14:26:45.284 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-24T14:26:45.285 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T14:26:45.285 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-24T14:26:45.285 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-24T14:26:45.285 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-24T14:26:45.285 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:18.0.0-1954.g1a671c19.el8.x86_64 22/25 2023-01-24T14:26:45.286 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T14:26:45.286 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:18.0.0-1954.g1a671c19.el8.x86_64 24/25 2023-01-24T14:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T14:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:26:46.118 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2023-01-24T14:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.119 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T14:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.120 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T14:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T14:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-24T14:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.121 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T14:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T14:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.122 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T14:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T14:26:46.123 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:26:46.124 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:26:46.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-test 2023-01-24T14:26:47.152 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:46 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:26:48.300 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:26:48.301 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:26:48.301 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:26:48.301 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:26:48.302 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:26:48.302 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 149 M 2023-01-24T14:26:48.302 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:26:48.302 INFO:teuthology.orchestra.run.smithi105.stdout: cryptopp x86_64 8.6.0-1.el8 epel 1.3 M 2023-01-24T14:26:48.303 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-24T14:26:48.303 INFO:teuthology.orchestra.run.smithi105.stdout: libasan6 x86_64 11.2.1-9.1.el8 CentOS-AppStream 402 k 2023-01-24T14:26:48.303 INFO:teuthology.orchestra.run.smithi105.stdout: libatomic x86_64 8.5.0-18.el8 CentOS-BaseOS 25 k 2023-01-24T14:26:48.303 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 134 k 2023-01-24T14:26:48.303 INFO:teuthology.orchestra.run.smithi105.stdout: libubsan x86_64 8.5.0-18.el8 CentOS-BaseOS 152 k 2023-01-24T14:26:48.304 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-24T14:26:48.304 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-24T14:26:48.304 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-24T14:26:48.304 INFO:teuthology.orchestra.run.smithi105.stdout: yaml-cpp x86_64 0.6.2-3.el8 epel 171 k 2023-01-24T14:26:48.304 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:26:48.305 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:26:48.305 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:26:48.305 INFO:teuthology.orchestra.run.smithi105.stdout:Install 11 Packages 2023-01-24T14:26:48.305 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:26:48.306 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 152 M 2023-01-24T14:26:48.306 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 1.7 G 2023-01-24T14:26:48.307 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:26:48.442 INFO:teuthology.orchestra.run.smithi105.stdout:(1/11): jq-1.6-6.el8.x86_64.rpm 1.4 MB/s | 202 kB 00:00 2023-01-24T14:26:48.542 INFO:teuthology.orchestra.run.smithi105.stdout:(2/11): libasan6-11.2.1-9.1.el8.x86_64.rpm 3.9 MB/s | 402 kB 00:00 2023-01-24T14:26:48.659 INFO:teuthology.orchestra.run.smithi105.stdout:(3/11): oniguruma-6.8.2-2.el8.x86_64.rpm 1.6 MB/s | 187 kB 00:00 2023-01-24T14:26:48.751 INFO:teuthology.orchestra.run.smithi105.stdout:(4/11): socat-1.7.4.1-1.el8.x86_64.rpm 3.4 MB/s | 323 kB 00:00 2023-01-24T14:26:48.777 INFO:teuthology.orchestra.run.smithi105.stdout:(5/11): libcephsqlite-18.0.0-1954.g1a671c19.el8 285 kB/s | 134 kB 00:00 2023-01-24T14:26:48.852 INFO:teuthology.orchestra.run.smithi105.stdout:(6/11): xmlstarlet-1.6.1-20.el8.x86_64.rpm 703 kB/s | 71 kB 00:00 2023-01-24T14:26:48.910 INFO:teuthology.orchestra.run.smithi105.stdout:(7/11): libatomic-8.5.0-18.el8.x86_64.rpm 188 kB/s | 25 kB 00:00 2023-01-24T14:26:48.944 INFO:teuthology.orchestra.run.smithi105.stdout:(8/11): libubsan-8.5.0-18.el8.x86_64.rpm 1.6 MB/s | 152 kB 00:00 2023-01-24T14:26:49.128 INFO:teuthology.orchestra.run.smithi105.stdout:(9/11): yaml-cpp-0.6.2-3.el8.x86_64.rpm 931 kB/s | 171 kB 00:00 2023-01-24T14:26:49.245 INFO:teuthology.orchestra.run.smithi105.stdout:(10/11): cryptopp-8.6.0-1.el8.x86_64.rpm 3.8 MB/s | 1.3 MB 00:00 2023-01-24T14:26:58.194 INFO:teuthology.orchestra.run.smithi105.stdout:(11/11): ceph-test-18.0.0-1954.g1a671c19.el8.x8 15 MB/s | 149 MB 00:09 2023-01-24T14:26:58.198 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:26:58.198 INFO:teuthology.orchestra.run.smithi105.stdout:Total 15 MB/s | 152 MB 00:09 2023-01-24T14:26:58.270 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:26:58.293 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:26:58.293 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:26:58.991 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:26:58.995 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:26:59.996 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:27:00.318 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : yaml-cpp-0.6.2-3.el8.x86_64 1/11 2023-01-24T14:27:00.525 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cryptopp-8.6.0-1.el8.x86_64 2/11 2023-01-24T14:27:00.680 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libubsan-8.5.0-18.el8.x86_64 3/11 2023-01-24T14:27:00.750 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libubsan-8.5.0-18.el8.x86_64 3/11 2023-01-24T14:27:00.847 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libatomic-8.5.0-18.el8.x86_64 4/11 2023-01-24T14:27:00.918 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libatomic-8.5.0-18.el8.x86_64 4/11 2023-01-24T14:27:01.167 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 5/11 2023-01-24T14:27:01.341 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 6/11 2023-01-24T14:27:01.571 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 7/11 2023-01-24T14:27:01.652 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 7/11 2023-01-24T14:27:01.858 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-6.el8.x86_64 8/11 2023-01-24T14:27:02.046 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-01-24T14:27:02.115 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libasan6-11.2.1-9.1.el8.x86_64 9/11 2023-01-24T14:27:02.263 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:18.0.0-1954.g1a671c19.el8.x86_64 10/11 2023-01-24T14:27:25.178 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1954.g1a671c19.el8.x86_64 10/11 2023-01-24T14:27:25.482 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:18.0.0-1954.g1a671c19.el8.x86_64 11/11 2023-01-24T14:27:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-test-2:18.0.0-1954.g1a671c19.el8.x86_64 11/11 2023-01-24T14:27:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:18.0.0-1954.g1a671c19.el8.x86_64 1/11 2023-01-24T14:27:25.785 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:18.0.0-1954.g1a671c19.el8.x86_64 2/11 2023-01-24T14:27:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-6.el8.x86_64 3/11 2023-01-24T14:27:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libasan6-11.2.1-9.1.el8.x86_64 4/11 2023-01-24T14:27:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/11 2023-01-24T14:27:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 6/11 2023-01-24T14:27:25.786 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 7/11 2023-01-24T14:27:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libatomic-8.5.0-18.el8.x86_64 8/11 2023-01-24T14:27:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libubsan-8.5.0-18.el8.x86_64 9/11 2023-01-24T14:27:25.787 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cryptopp-8.6.0-1.el8.x86_64 10/11 2023-01-24T14:27:26.382 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : yaml-cpp-0.6.2-3.el8.x86_64 11/11 2023-01-24T14:27:26.383 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:27:26.383 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:27:26.383 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:27:26.383 INFO:teuthology.orchestra.run.smithi105.stdout: cryptopp-8.6.0-1.el8.x86_64 2023-01-24T14:27:26.383 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-6.el8.x86_64 2023-01-24T14:27:26.384 INFO:teuthology.orchestra.run.smithi105.stdout: libasan6-11.2.1-9.1.el8.x86_64 2023-01-24T14:27:26.384 INFO:teuthology.orchestra.run.smithi105.stdout: libatomic-8.5.0-18.el8.x86_64 2023-01-24T14:27:26.384 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:27:26.384 INFO:teuthology.orchestra.run.smithi105.stdout: libubsan-8.5.0-18.el8.x86_64 2023-01-24T14:27:26.385 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T14:27:26.385 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-24T14:27:26.385 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T14:27:26.385 INFO:teuthology.orchestra.run.smithi105.stdout: yaml-cpp-0.6.2-3.el8.x86_64 2023-01-24T14:27:26.385 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:27:26.385 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:27:26.627 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph 2023-01-24T14:27:27.358 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:01:26 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:27:28.502 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:27:28.505 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2023-01-24T14:27:28.505 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:27:28.505 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2023-01-24T14:27:28.505 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:27:28.506 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 6.4 k 2023-01-24T14:27:28.506 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:27:28.506 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 2.2 M 2023-01-24T14:27:28.506 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 1.5 M 2023-01-24T14:27:28.507 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 236 k 2023-01-24T14:27:28.507 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 7.4 M 2023-01-24T14:27:28.507 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 151 M 2023-01-24T14:27:28.507 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T14:27:28.508 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T14:27:28.508 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-24T14:27:28.508 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T14:27:28.508 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T14:27:28.508 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-24T14:27:28.509 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T14:27:28.509 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T14:27:28.509 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T14:27:28.509 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-24T14:27:28.509 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-24T14:27:28.510 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T14:27:28.510 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T14:27:28.510 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T14:27:28.510 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-24T14:27:28.510 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-24T14:27:28.511 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-24T14:27:28.511 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T14:27:28.511 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T14:27:28.511 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T14:27:28.511 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T14:27:28.512 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T14:27:28.512 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T14:27:28.512 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T14:27:28.512 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-24T14:27:28.512 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T14:27:28.513 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:27:28.513 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:27:28.513 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================= 2023-01-24T14:27:28.514 INFO:teuthology.orchestra.run.smithi105.stdout:Install 32 Packages 2023-01-24T14:27:28.514 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:27:28.515 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 165 M 2023-01-24T14:27:28.515 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 2.2 G 2023-01-24T14:27:28.515 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:27:28.850 INFO:teuthology.orchestra.run.smithi105.stdout:(1/32): ceph-18.0.0-1954.g1a671c19.el8.x86_64.r 19 kB/s | 6.4 kB 00:00 2023-01-24T14:27:30.558 INFO:teuthology.orchestra.run.smithi105.stdout:(2/32): ceph-mgr-18.0.0-1954.g1a671c19.el8.x86_ 738 kB/s | 1.5 MB 00:02 2023-01-24T14:27:31.292 INFO:teuthology.orchestra.run.smithi105.stdout:(3/32): ceph-mds-18.0.0-1954.g1a671c19.el8.x86_ 803 kB/s | 2.2 MB 00:02 2023-01-24T14:27:31.567 INFO:teuthology.orchestra.run.smithi105.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1954.g1a67 858 kB/s | 236 kB 00:00 2023-01-24T14:27:31.684 INFO:teuthology.orchestra.run.smithi105.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-01-24T14:27:31.734 INFO:teuthology.orchestra.run.smithi105.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 776 kB/s | 39 kB 00:00 2023-01-24T14:27:31.793 INFO:teuthology.orchestra.run.smithi105.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 1.7 MB/s | 103 kB 00:00 2023-01-24T14:27:31.843 INFO:teuthology.orchestra.run.smithi105.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-01-24T14:27:31.927 INFO:teuthology.orchestra.run.smithi105.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 5.4 MB/s | 457 kB 00:00 2023-01-24T14:27:31.977 INFO:teuthology.orchestra.run.smithi105.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 237 kB 00:00 2023-01-24T14:27:32.070 INFO:teuthology.orchestra.run.smithi105.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 6.0 MB/s | 559 kB 00:00 2023-01-24T14:27:32.145 INFO:teuthology.orchestra.run.smithi105.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 1.4 MB/s | 109 kB 00:00 2023-01-24T14:27:32.287 INFO:teuthology.orchestra.run.smithi105.stdout:(13/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 308 kB/s | 44 kB 00:00 2023-01-24T14:27:32.395 INFO:teuthology.orchestra.run.smithi105.stdout:(14/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.7 MB/s | 185 kB 00:00 2023-01-24T14:27:32.464 INFO:teuthology.orchestra.run.smithi105.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.4 MB/s | 173 kB 00:00 2023-01-24T14:27:32.556 INFO:teuthology.orchestra.run.smithi105.stdout:(16/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.2 MB/s | 384 kB 00:00 2023-01-24T14:27:32.596 INFO:teuthology.orchestra.run.smithi105.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2023-01-24T14:27:32.638 INFO:teuthology.orchestra.run.smithi105.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 423 kB/s | 18 kB 00:00 2023-01-24T14:27:32.688 INFO:teuthology.orchestra.run.smithi105.stdout:(19/32): python3-logutils-0.3.5-11.el8.noarch.r 974 kB/s | 49 kB 00:00 2023-01-24T14:27:32.739 INFO:teuthology.orchestra.run.smithi105.stdout:(20/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2023-01-24T14:27:32.800 INFO:teuthology.orchestra.run.smithi105.stdout:(21/32): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2023-01-24T14:27:32.856 INFO:teuthology.orchestra.run.smithi105.stdout:(22/32): python3-portend-2.6-1.el8.noarch.rpm 267 kB/s | 16 kB 00:00 2023-01-24T14:27:32.915 INFO:teuthology.orchestra.run.smithi105.stdout:(23/32): python3-simplegeneric-0.8.1-17.el8.noa 321 kB/s | 19 kB 00:00 2023-01-24T14:27:32.965 INFO:teuthology.orchestra.run.smithi105.stdout:(24/32): python3-singledispatch-3.4.0.3-18.el8. 479 kB/s | 24 kB 00:00 2023-01-24T14:27:33.015 INFO:teuthology.orchestra.run.smithi105.stdout:(25/32): python3-tempora-1.14.1-5.el8.noarch.rp 584 kB/s | 29 kB 00:00 2023-01-24T14:27:33.066 INFO:teuthology.orchestra.run.smithi105.stdout:(26/32): python3-trustme-0.6.0-4.el8.noarch.rpm 552 kB/s | 27 kB 00:00 2023-01-24T14:27:33.124 INFO:teuthology.orchestra.run.smithi105.stdout:(27/32): python3-waitress-1.2.1-2.el8.1.noarch. 4.0 MB/s | 241 kB 00:00 2023-01-24T14:27:33.183 INFO:teuthology.orchestra.run.smithi105.stdout:(28/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.2 MB/s | 251 kB 00:00 2023-01-24T14:27:33.233 INFO:teuthology.orchestra.run.smithi105.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.7 MB/s | 85 kB 00:00 2023-01-24T14:27:33.305 INFO:teuthology.orchestra.run.smithi105.stdout:(30/32): python3-zc-lockfile-2.0-2.el8.noarch.r 321 kB/s | 23 kB 00:00 2023-01-24T14:27:34.344 INFO:teuthology.orchestra.run.smithi105.stdout:(31/32): ceph-mon-18.0.0-1954.g1a671c19.el8.x86 1.3 MB/s | 7.4 MB 00:05 2023-01-24T14:38:09.376 INFO:teuthology.orchestra.run.smithi105.stdout:[MIRROR] ceph-osd-18.0.0-1954.g1a671c19.el8.x86_64.rpm: Curl error (18): Transferred a partial file for https://1.chacra.ceph.com/r/ceph/main/1a671c198559f54389cd5afa22085a0c7b29a1ef/centos/8/flavors/crimson/x86_64/ceph-osd-18.0.0-1954.g1a671c19.el8.x86_64.rpm [transfer closed with 11258989 bytes remaining to read] 2023-01-24T14:39:34.387 INFO:teuthology.orchestra.run.smithi105.stdout:(32/32): ceph-osd-18.0.0-1954.g1a671c19.el8.x86 213 kB/s | 151 MB 12:03 2023-01-24T14:39:34.388 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:39:34.388 INFO:teuthology.orchestra.run.smithi105.stdout:Total 233 kB/s | 165 MB 12:05 2023-01-24T14:39:34.464 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:39:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:39:34.503 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:39:35.398 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:39:35.401 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:39:37.268 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:39:37.509 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-24T14:39:37.770 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-24T14:39:37.984 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-24T14:39:38.252 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-24T14:39:38.408 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-24T14:39:38.630 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-24T14:39:38.807 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-24T14:39:38.976 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-24T14:39:39.225 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-24T14:39:39.547 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-24T14:39:39.750 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-24T14:39:39.987 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-24T14:39:40.370 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-24T14:39:40.640 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-24T14:39:40.862 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-24T14:39:41.086 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-24T14:39:41.330 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-24T14:39:41.660 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-24T14:39:41.859 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-24T14:39:42.081 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-24T14:39:42.292 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-24T14:39:42.593 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-24T14:39:42.899 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-24T14:39:43.122 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-24T14:39:43.508 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-24T14:39:43.901 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-24T14:39:44.339 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1954.g1a671c19.el8. 27/32 2023-01-24T14:39:44.567 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:18.0.0-1954.g1a671c19.el8.x86_64 28/32 2023-01-24T14:40:11.492 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1954.g1a671c19.el8.x86_64 28/32 2023-01-24T14:40:11.682 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:18.0.0-1954.g1a671c19.el8.x86_64 29/32 2023-01-24T14:40:12.435 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:18.0.0-1954.g1a671c19.el8.x86_64 29/32 2023-01-24T14:40:12.708 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:18.0.0-1954.g1a671c19.el8.x86_64 30/32 2023-01-24T14:40:12.925 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:18.0.0-1954.g1a671c19.el8.x86_64 30/32 2023-01-24T14:40:13.150 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:18.0.0-1954.g1a671c19.el8.x86_64 31/32 2023-01-24T14:40:13.167 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:18.0.0-1954.g1a671c19.el8.x86_64 31/32 2023-01-24T14:40:13.339 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:18.0.0-1954.g1a671c19.el8.x86_64 32/32 2023-01-24T14:40:13.775 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-2:18.0.0-1954.g1a671c19.el8.x86_64 32/32 2023-01-24T14:40:13.775 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:18.0.0-1954.g1a671c19.el8.x86_64 1/32 2023-01-24T14:40:13.775 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:18.0.0-1954.g1a671c19.el8.x86_64 2/32 2023-01-24T14:40:13.775 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:18.0.0-1954.g1a671c19.el8.x86_64 3/32 2023-01-24T14:40:13.776 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:18.0.0-1954.g1a671c19.el8.x86_64 4/32 2023-01-24T14:40:13.776 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:18.0.0-1954.g1a671c19.el8.x86_64 5/32 2023-01-24T14:40:13.776 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1954.g1a671c19.el8. 6/32 2023-01-24T14:40:13.776 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-24T14:40:13.776 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-24T14:40:13.777 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-24T14:40:13.777 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-24T14:40:13.777 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-24T14:40:13.777 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-24T14:40:13.777 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-24T14:40:13.778 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-24T14:40:13.778 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-24T14:40:13.778 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-24T14:40:13.778 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-24T14:40:13.778 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-24T14:40:13.779 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-24T14:40:13.779 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-24T14:40:13.779 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-24T14:40:13.779 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-24T14:40:13.779 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-24T14:40:13.780 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-24T14:40:13.780 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-24T14:40:13.780 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-24T14:40:13.780 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-24T14:40:13.781 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-24T14:40:13.781 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-24T14:40:13.781 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-24T14:40:13.781 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-24T14:40:14.805 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-24T14:40:14.805 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:14.806 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:40:14.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:40:14.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:40:14.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:40:14.806 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:40:14.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:40:14.807 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:40:14.807 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T14:40:14.807 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T14:40:14.807 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-24T14:40:14.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T14:40:14.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T14:40:14.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-24T14:40:14.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T14:40:14.808 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T14:40:14.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T14:40:14.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T14:40:14.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T14:40:14.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T14:40:14.809 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T14:40:14.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T14:40:14.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-24T14:40:14.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-24T14:40:14.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T14:40:14.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T14:40:14.810 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T14:40:14.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T14:40:14.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T14:40:14.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T14:40:14.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T14:40:14.811 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T14:40:14.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T14:40:14.812 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T14:40:14.812 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:14.812 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:40:15.050 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-base 2023-01-24T14:40:15.767 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:14:14 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:40:16.813 INFO:teuthology.orchestra.run.smithi105.stdout:Package ceph-base-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:40:16.913 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:40:16.914 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:40:16.914 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:40:17.380 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install cephadm 2023-01-24T14:40:18.097 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:14:17 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:40:19.225 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:40:19.226 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:19.226 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:40:19.226 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:19.226 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:40:19.227 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 82 k 2023-01-24T14:40:19.227 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:19.227 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:40:19.227 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:19.227 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:40:19.228 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:19.228 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 82 k 2023-01-24T14:40:19.228 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 374 k 2023-01-24T14:40:19.228 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:40:19.657 INFO:teuthology.orchestra.run.smithi105.stdout:cephadm-18.0.0-1954.g1a671c19.el8.noarch.rpm 191 kB/s | 82 kB 00:00 2023-01-24T14:40:19.659 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:40:19.659 INFO:teuthology.orchestra.run.smithi105.stdout:Total 190 kB/s | 82 kB 00:00 2023-01-24T14:40:19.659 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:40:19.667 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:40:19.667 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:40:19.669 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:40:19.673 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:40:20.239 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:40:29.932 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 1/1 2023-01-24T14:40:31.879 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 1/1 2023-01-24T14:40:32.119 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 1/1 2023-01-24T14:40:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 1/1 2023-01-24T14:40:32.859 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:32.859 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:40:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:40:32.860 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:32.860 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:40:33.111 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T14:40:33.843 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:14:32 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:40:34.992 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:40:34.993 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:34.993 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T14:40:34.993 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:34.994 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:40:34.994 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 157 k 2023-01-24T14:40:34.994 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:34.994 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:40:34.995 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:34.995 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:40:34.995 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:34.995 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 157 k 2023-01-24T14:40:34.996 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 540 k 2023-01-24T14:40:34.996 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:40:35.512 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-immutable-object-cache-18.0.0-1954.g1a671c 303 kB/s | 157 kB 00:00 2023-01-24T14:40:35.513 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:40:35.513 INFO:teuthology.orchestra.run.smithi105.stdout:Total 302 kB/s | 157 kB 00:00 2023-01-24T14:40:35.514 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:40:35.525 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:40:35.525 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:40:35.574 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:40:35.578 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:40:35.704 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:40:35.864 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1954.g1a671c19. 1/1 2023-01-24T14:40:36.298 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1954.g1a671c19. 1/1 2023-01-24T14:40:36.798 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1954.g1a671c19. 1/1 2023-01-24T14:40:36.799 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:36.799 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:40:36.800 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:40:36.800 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:36.800 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:40:37.002 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr 2023-01-24T14:40:37.728 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:14:36 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:40:38.779 INFO:teuthology.orchestra.run.smithi105.stdout:Package ceph-mgr-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:40:38.879 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:40:38.880 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:40:38.881 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:40:38.983 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T14:40:39.695 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:14:38 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:40:40.843 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:40:40.844 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:40.844 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:40:40.844 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:40.844 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:40:40.845 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 1.7 M 2023-01-24T14:40:40.845 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:40:40.845 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 23 k 2023-01-24T14:40:40.845 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 15 k 2023-01-24T14:40:40.846 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-24T14:40:40.846 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T14:40:40.846 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T14:40:40.846 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:40.846 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:40:40.846 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:40:40.847 INFO:teuthology.orchestra.run.smithi105.stdout:Install 6 Packages 2023-01-24T14:40:40.847 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:40.848 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 2.0 M 2023-01-24T14:40:40.848 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 8.2 M 2023-01-24T14:40:40.848 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:40:41.107 INFO:teuthology.orchestra.run.smithi105.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1954.g1a6 88 kB/s | 23 kB 00:00 2023-01-24T14:40:41.132 INFO:teuthology.orchestra.run.smithi105.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1954.g1a67 51 kB/s | 15 kB 00:00 2023-01-24T14:40:41.191 INFO:teuthology.orchestra.run.smithi105.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 510 kB/s | 43 kB 00:00 2023-01-24T14:40:41.333 INFO:teuthology.orchestra.run.smithi105.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 168 kB/s | 34 kB 00:00 2023-01-24T14:40:41.458 INFO:teuthology.orchestra.run.smithi105.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 736 kB/s | 196 kB 00:00 2023-01-24T14:40:41.850 INFO:teuthology.orchestra.run.smithi105.stdout:(6/6): ceph-mgr-dashboard-18.0.0-1954.g1a671c19 1.7 MB/s | 1.7 MB 00:01 2023-01-24T14:40:41.852 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:40:41.853 INFO:teuthology.orchestra.run.smithi105.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2023-01-24T14:40:41.863 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:40:41.871 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:40:41.872 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:40:41.953 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:40:41.957 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:40:42.217 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:40:42.467 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-24T14:40:42.721 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-24T14:40:42.906 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-24T14:40:43.894 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1954.g1a671c19.el8.n 4/6 2023-01-24T14:40:44.313 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1954.g1a671c19.el8. 5/6 2023-01-24T14:40:44.594 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1954.g1a671c19.el8.noarc 6/6 2023-01-24T14:40:44.875 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1954.g1a671c19.el8.noarc 6/6 2023-01-24T14:40:44.875 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1954.g1a671c19.el8. 1/6 2023-01-24T14:40:44.876 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1954.g1a671c19.el8.noarc 2/6 2023-01-24T14:40:44.876 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1954.g1a671c19.el8.n 3/6 2023-01-24T14:40:44.876 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-24T14:40:44.876 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-24T14:40:45.381 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-24T14:40:45.382 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:45.382 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:40:45.382 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:40:45.382 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:40:45.382 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:40:45.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T14:40:45.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T14:40:45.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T14:40:45.383 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:45.383 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:40:45.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T14:40:46.219 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:14:45 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:40:47.379 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:40:47.380 INFO:teuthology.orchestra.run.smithi105.stdout:===================================================================================================== 2023-01-24T14:40:47.380 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:40:47.381 INFO:teuthology.orchestra.run.smithi105.stdout:===================================================================================================== 2023-01-24T14:40:47.381 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:40:47.381 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 7.4 M 2023-01-24T14:40:47.381 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:40:47.381 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-24T14:40:47.382 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-24T14:40:47.382 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-24T14:40:47.382 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-24T14:40:47.382 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-24T14:40:47.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-24T14:40:47.383 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-24T14:40:47.383 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:47.383 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:40:47.383 INFO:teuthology.orchestra.run.smithi105.stdout:===================================================================================================== 2023-01-24T14:40:47.384 INFO:teuthology.orchestra.run.smithi105.stdout:Install 8 Packages 2023-01-24T14:40:47.384 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:47.384 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 35 M 2023-01-24T14:40:47.385 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 210 M 2023-01-24T14:40:47.385 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:40:47.557 INFO:teuthology.orchestra.run.smithi105.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 27 MB/s | 4.7 MB 00:00 2023-01-24T14:40:47.657 INFO:teuthology.orchestra.run.smithi105.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 37 MB/s | 3.7 MB 00:00 2023-01-24T14:40:47.691 INFO:teuthology.orchestra.run.smithi105.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 6.6 MB/s | 225 kB 00:00 2023-01-24T14:40:47.783 INFO:teuthology.orchestra.run.smithi105.stdout:(4/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.9 MB 00:00 2023-01-24T14:40:47.833 INFO:teuthology.orchestra.run.smithi105.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 13 MB/s | 645 kB 00:00 2023-01-24T14:40:47.892 INFO:teuthology.orchestra.run.smithi105.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 2.9 MB/s | 172 kB 00:00 2023-01-24T14:40:48.084 INFO:teuthology.orchestra.run.smithi105.stdout:(7/8): python3-scipy-1.0.0-21.module_el8.5.0+77 35 MB/s | 14 MB 00:00 2023-01-24T14:40:48.376 INFO:teuthology.orchestra.run.smithi105.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-195 7.4 MB/s | 7.4 MB 00:00 2023-01-24T14:40:48.378 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:40:48.378 INFO:teuthology.orchestra.run.smithi105.stdout:Total 36 MB/s | 35 MB 00:00 2023-01-24T14:40:48.610 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:40:48.628 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:40:48.628 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:40:49.039 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:40:49.043 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:40:49.653 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:40:49.761 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-24T14:40:49.891 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-24T14:40:50.086 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-24T14:40:50.669 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-24T14:40:50.870 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-24T14:40:51.487 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-24T14:40:51.628 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-24T14:40:52.339 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-24T14:40:52.732 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-24T14:40:52.977 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-24T14:40:54.757 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-24T14:40:56.000 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-24T14:40:56.210 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1954.g1a671c1 8/8 2023-01-24T14:40:56.564 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1954.g1a671c1 8/8 2023-01-24T14:40:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1954.g1a671c1 1/8 2023-01-24T14:40:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-24T14:40:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-24T14:40:56.565 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-24T14:40:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-24T14:40:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-24T14:40:56.566 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-24T14:40:57.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-24T14:40:57.046 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:57.046 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:40:57.046 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:40:57.046 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-24T14:40:57.046 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-24T14:40:57.047 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-24T14:40:57.047 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-24T14:40:57.047 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-24T14:40:57.047 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-24T14:40:57.048 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-24T14:40:57.048 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:57.048 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:40:57.163 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-rook 2023-01-24T14:40:57.890 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:14:56 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:40:59.033 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:40:59.034 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================== 2023-01-24T14:40:59.035 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:40:59.035 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================== 2023-01-24T14:40:59.035 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:40:59.035 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 47 k 2023-01-24T14:40:59.035 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:40:59.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T14:40:59.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T14:40:59.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T14:40:59.036 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-24T14:40:59.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-24T14:40:59.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T14:40:59.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-24T14:40:59.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-24T14:40:59.037 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-24T14:40:59.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-24T14:40:59.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T14:40:59.038 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T14:40:59.038 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:59.038 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:40:59.039 INFO:teuthology.orchestra.run.smithi105.stdout:======================================================================================== 2023-01-24T14:40:59.039 INFO:teuthology.orchestra.run.smithi105.stdout:Install 13 Packages 2023-01-24T14:40:59.039 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:40:59.040 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 1.9 M 2023-01-24T14:40:59.040 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 25 M 2023-01-24T14:40:59.040 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:40:59.115 INFO:teuthology.orchestra.run.smithi105.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 345 kB/s | 27 kB 00:00 2023-01-24T14:40:59.140 INFO:teuthology.orchestra.run.smithi105.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 195 kB/s | 20 kB 00:00 2023-01-24T14:40:59.166 INFO:teuthology.orchestra.run.smithi105.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-24T14:40:59.191 INFO:teuthology.orchestra.run.smithi105.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-24T14:40:59.217 INFO:teuthology.orchestra.run.smithi105.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-24T14:40:59.275 INFO:teuthology.orchestra.run.smithi105.stdout:(6/13): ceph-mgr-rook-18.0.0-1954.g1a671c19.el8 200 kB/s | 47 kB 00:00 2023-01-24T14:40:59.300 INFO:teuthology.orchestra.run.smithi105.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 390 kB/s | 43 kB 00:00 2023-01-24T14:40:59.393 INFO:teuthology.orchestra.run.smithi105.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 188 kB/s | 33 kB 00:00 2023-01-24T14:40:59.418 INFO:teuthology.orchestra.run.smithi105.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 114 kB/s | 16 kB 00:00 2023-01-24T14:40:59.443 INFO:teuthology.orchestra.run.smithi105.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 571 kB/s | 81 kB 00:00 2023-01-24T14:40:59.494 INFO:teuthology.orchestra.run.smithi105.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 812 kB/s | 61 kB 00:00 2023-01-24T14:40:59.519 INFO:teuthology.orchestra.run.smithi105.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 810 kB/s | 61 kB 00:00 2023-01-24T14:40:59.919 INFO:teuthology.orchestra.run.smithi105.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 2.2 MB/s | 1.2 MB 00:00 2023-01-24T14:40:59.920 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:40:59.921 INFO:teuthology.orchestra.run.smithi105.stdout:Total 2.2 MB/s | 1.9 MB 00:00 2023-01-24T14:40:59.975 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:40:59.990 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:40:59.990 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:00.090 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:00.093 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:00.601 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:00.845 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T14:41:01.052 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-24T14:41:01.295 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-24T14:41:01.541 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-24T14:41:01.723 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-24T14:41:02.021 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-24T14:41:02.362 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-24T14:41:02.589 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T14:41:03.639 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T14:41:03.974 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-24T14:41:04.191 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-24T14:41:04.410 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-24T14:41:04.826 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:18.0.0-1954.g1a671c19.el8.noarch 13/13 2023-01-24T14:41:05.138 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1954.g1a671c19.el8.noarch 13/13 2023-01-24T14:41:05.139 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1954.g1a671c19.el8.noarch 1/13 2023-01-24T14:41:05.139 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-24T14:41:05.139 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-24T14:41:05.139 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-24T14:41:05.139 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-24T14:41:05.139 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T14:41:05.140 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T14:41:05.140 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-24T14:41:05.140 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T14:41:05.140 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-24T14:41:05.140 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-24T14:41:05.141 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-24T14:41:05.692 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-24T14:41:05.693 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:05.693 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:05.693 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:41:05.693 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T14:41:05.694 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T14:41:05.694 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T14:41:05.694 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T14:41:05.694 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T14:41:05.694 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T14:41:05.695 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T14:41:05.695 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T14:41:05.695 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T14:41:05.695 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T14:41:05.695 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T14:41:05.696 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T14:41:05.696 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:05.696 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:05.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T14:41:06.575 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:05 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:07.709 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:07.710 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:07.710 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:41:07.711 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:07.711 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:07.711 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 120 k 2023-01-24T14:41:07.711 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:41:07.711 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T14:41:07.712 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-24T14:41:07.712 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-24T14:41:07.712 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T14:41:07.712 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:07.712 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:07.713 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:07.713 INFO:teuthology.orchestra.run.smithi105.stdout:Install 5 Packages 2023-01-24T14:41:07.713 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:07.714 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 6.0 M 2023-01-24T14:41:07.714 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 26 M 2023-01-24T14:41:07.714 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:07.786 INFO:teuthology.orchestra.run.smithi105.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 7.2 MB/s | 538 kB 00:00 2023-01-24T14:41:07.886 INFO:teuthology.orchestra.run.smithi105.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 28 MB/s | 4.8 MB 00:00 2023-01-24T14:41:07.928 INFO:teuthology.orchestra.run.smithi105.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1954.g1a671c19.e 556 kB/s | 120 kB 00:00 2023-01-24T14:41:08.045 INFO:teuthology.orchestra.run.smithi105.stdout:(4/5): python3-natsort-7.1.1-2.el8.noarch.rpm 380 kB/s | 60 kB 00:00 2023-01-24T14:41:08.120 INFO:teuthology.orchestra.run.smithi105.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.4 MB/s | 476 kB 00:00 2023-01-24T14:41:08.122 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:08.122 INFO:teuthology.orchestra.run.smithi105.stdout:Total 14 MB/s | 6.0 MB 00:00 2023-01-24T14:41:08.174 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:08.184 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:08.185 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:08.328 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:08.331 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:08.642 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:08.972 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-24T14:41:09.933 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T14:41:10.361 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-24T14:41:10.637 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-24T14:41:10.875 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 5/5 2023-01-24T14:41:11.174 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 5/5 2023-01-24T14:41:11.175 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 1/5 2023-01-24T14:41:11.175 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-24T14:41:11.175 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-24T14:41:11.175 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T14:41:11.674 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-24T14:41:11.675 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:11.675 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:11.675 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:41:11.675 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T14:41:11.675 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-24T14:41:11.676 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-24T14:41:11.676 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T14:41:11.676 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:11.676 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:11.792 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-fuse 2023-01-24T14:41:12.517 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:11 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:13.665 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:13.666 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:13.667 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:41:13.667 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:13.667 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:13.667 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 814 k 2023-01-24T14:41:13.668 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2023-01-24T14:41:13.668 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-24T14:41:13.668 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:13.669 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:13.669 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:13.669 INFO:teuthology.orchestra.run.smithi105.stdout:Install 2 Packages 2023-01-24T14:41:13.670 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:13.670 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 897 k 2023-01-24T14:41:13.671 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 3.3 M 2023-01-24T14:41:13.671 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:13.737 INFO:teuthology.orchestra.run.smithi105.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2023-01-24T14:41:14.220 INFO:teuthology.orchestra.run.smithi105.stdout:(2/2): ceph-fuse-18.0.0-1954.g1a671c19.el8.x86_ 1.4 MB/s | 814 kB 00:00 2023-01-24T14:41:14.221 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:14.221 INFO:teuthology.orchestra.run.smithi105.stdout:Total 1.6 MB/s | 897 kB 00:00 2023-01-24T14:41:14.231 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:14.252 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:14.252 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:14.346 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:14.349 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:14.558 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:14.872 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-24T14:41:15.073 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 2/2 2023-01-24T14:41:15.511 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 2/2 2023-01-24T14:41:15.511 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 1/2 2023-01-24T14:41:16.366 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-24T14:41:16.367 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:16.367 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:16.367 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-24T14:41:16.367 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:16.368 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:16.582 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-volume 2023-01-24T14:41:17.329 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:16 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:18.443 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:18.444 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:18.444 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2023-01-24T14:41:18.444 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:18.445 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:18.445 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:18.0.0-1954.g1a671c19.el8 ceph-noarch 263 k 2023-01-24T14:41:18.445 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:18.445 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:18.446 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:18.446 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:41:18.446 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:18.446 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 263 k 2023-01-24T14:41:18.446 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 1.3 M 2023-01-24T14:41:18.447 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:18.850 INFO:teuthology.orchestra.run.smithi105.stdout:ceph-volume-18.0.0-1954.g1a671c19.el8.noarch.rp 650 kB/s | 263 kB 00:00 2023-01-24T14:41:18.850 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:18.851 INFO:teuthology.orchestra.run.smithi105.stdout:Total 648 kB/s | 263 kB 00:00 2023-01-24T14:41:18.851 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:18.861 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:18.861 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:18.936 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:18.939 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:19.119 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:19.408 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:18.0.0-1954.g1a671c19.el8.noarch 1/1 2023-01-24T14:41:19.887 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:18.0.0-1954.g1a671c19.el8.noarch 1/1 2023-01-24T14:41:20.419 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:18.0.0-1954.g1a671c19.el8.noarch 1/1 2023-01-24T14:41:20.420 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:20.420 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:20.420 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:18.0.0-1954.g1a671c19.el8.noarch 2023-01-24T14:41:20.421 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:20.421 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:20.631 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install librados-devel 2023-01-24T14:41:21.368 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:20 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:22.516 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:22.516 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T14:41:22.517 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:22.517 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:22.517 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 139 k 2023-01-24T14:41:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:22.518 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:22.518 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:22.518 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:41:22.518 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:22.519 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 139 k 2023-01-24T14:41:22.521 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 512 k 2023-01-24T14:41:22.521 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:23.064 INFO:teuthology.orchestra.run.smithi105.stdout:librados-devel-18.0.0-1954.g1a671c19.el8.x86_64 254 kB/s | 139 kB 00:00 2023-01-24T14:41:23.064 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:23.065 INFO:teuthology.orchestra.run.smithi105.stdout:Total 253 kB/s | 139 kB 00:00 2023-01-24T14:41:23.065 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:23.070 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:23.070 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:23.123 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:23.127 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:23.250 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:23.496 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:23.796 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:24.275 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:24.276 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:24.276 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:24.276 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:41:24.277 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:24.277 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:24.515 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install libcephfs2 2023-01-24T14:41:25.245 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:24 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:26.281 INFO:teuthology.orchestra.run.smithi105.stdout:Package libcephfs2-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:41:26.383 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:26.385 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:41:26.385 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:26.499 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install libcephfs-devel 2023-01-24T14:41:27.217 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:26 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:28.361 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:28.362 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:28.362 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T14:41:28.362 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:28.363 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:28.363 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 30 k 2023-01-24T14:41:28.363 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:28.363 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:28.363 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:28.364 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:41:28.364 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:28.364 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 30 k 2023-01-24T14:41:28.364 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 137 k 2023-01-24T14:41:28.364 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:28.721 INFO:teuthology.orchestra.run.smithi105.stdout:libcephfs-devel-18.0.0-1954.g1a671c19.el8.x86_6 85 kB/s | 30 kB 00:00 2023-01-24T14:41:28.722 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:28.722 INFO:teuthology.orchestra.run.smithi105.stdout:Total 85 kB/s | 30 kB 00:00 2023-01-24T14:41:28.722 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:28.725 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:28.726 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:28.748 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:28.751 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:28.814 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:29.062 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:29.315 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:29.770 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:29.771 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:29.771 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:29.771 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:41:29.771 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:29.772 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:29.896 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install librados2 2023-01-24T14:41:30.612 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:29 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:31.642 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:41:31.743 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:31.744 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:41:31.745 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:31.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install librbd1 2023-01-24T14:41:32.608 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:31 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:33.661 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:41:33.763 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:33.765 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:41:33.765 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:33.868 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-rados 2023-01-24T14:41:34.594 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:33 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:35.636 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-rados-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:41:35.736 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:35.738 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:41:35.738 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:35.843 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-rgw 2023-01-24T14:41:36.560 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:35 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:37.596 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-rgw-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:41:37.698 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:37.699 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:41:37.699 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:37.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-cephfs 2023-01-24T14:41:38.629 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:37 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:39.674 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-cephfs-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:41:39.774 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:39.775 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:41:39.776 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:39.954 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install python3-rbd 2023-01-24T14:41:40.670 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:39 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:41.707 INFO:teuthology.orchestra.run.smithi105.stdout:Package python3-rbd-2:18.0.0-1954.g1a671c19.el8.x86_64 is already installed. 2023-01-24T14:41:41.809 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:41.810 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2023-01-24T14:41:41.810 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:41.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install rbd-fuse 2023-01-24T14:41:42.634 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:41 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:43.778 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:43.779 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:43.779 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2023-01-24T14:41:43.780 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:43.780 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:43.780 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 91 k 2023-01-24T14:41:43.780 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:43.781 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:43.781 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:43.781 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:41:43.781 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:43.781 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 91 k 2023-01-24T14:41:43.782 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 303 k 2023-01-24T14:41:43.782 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:44.094 INFO:teuthology.orchestra.run.smithi105.stdout:rbd-fuse-18.0.0-1954.g1a671c19.el8.x86_64.rpm 289 kB/s | 91 kB 00:00 2023-01-24T14:41:44.095 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:44.096 INFO:teuthology.orchestra.run.smithi105.stdout:Total 288 kB/s | 91 kB 00:00 2023-01-24T14:41:44.096 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:44.105 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:44.106 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:44.159 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:44.163 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:44.286 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:44.510 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:44.790 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:45.336 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:45.336 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:45.337 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:45.337 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:41:45.337 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:45.337 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:45.546 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install rbd-mirror 2023-01-24T14:41:46.277 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:45 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:47.420 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:47.421 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:47.422 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repo Size 2023-01-24T14:41:47.422 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:47.423 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:47.424 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 3.3 M 2023-01-24T14:41:47.424 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:47.424 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:47.425 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:47.425 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:41:47.425 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:47.425 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 3.3 M 2023-01-24T14:41:47.426 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 18 M 2023-01-24T14:41:47.426 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:48.114 INFO:teuthology.orchestra.run.smithi105.stdout:rbd-mirror-18.0.0-1954.g1a671c19.el8.x86_64.rpm 4.8 MB/s | 3.3 MB 00:00 2023-01-24T14:41:48.115 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:48.115 INFO:teuthology.orchestra.run.smithi105.stdout:Total 4.8 MB/s | 3.3 MB 00:00 2023-01-24T14:41:48.116 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:48.126 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:48.127 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:48.189 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:48.193 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:48.643 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:48.837 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:49.299 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:49.817 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:49.818 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:50.033 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install rbd-nbd 2023-01-24T14:41:50.770 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:15:49 ago on Tue 24 Jan 2023 02:26:01 PM UTC. 2023-01-24T14:41:51.901 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2023-01-24T14:41:51.903 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:51.903 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2023-01-24T14:41:51.903 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:51.903 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2023-01-24T14:41:51.903 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:18.0.0-1954.g1a671c19.el8 ceph 177 k 2023-01-24T14:41:51.904 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:51.904 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2023-01-24T14:41:51.904 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2023-01-24T14:41:51.904 INFO:teuthology.orchestra.run.smithi105.stdout:Install 1 Package 2023-01-24T14:41:51.904 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:51.905 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 177 k 2023-01-24T14:41:51.905 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 626 k 2023-01-24T14:41:51.905 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2023-01-24T14:41:52.229 INFO:teuthology.orchestra.run.smithi105.stdout:rbd-nbd-18.0.0-1954.g1a671c19.el8.x86_64.rpm 543 kB/s | 177 kB 00:00 2023-01-24T14:41:52.230 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2023-01-24T14:41:52.230 INFO:teuthology.orchestra.run.smithi105.stdout:Total 541 kB/s | 177 kB 00:00 2023-01-24T14:41:52.230 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2023-01-24T14:41:52.241 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2023-01-24T14:41:52.241 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2023-01-24T14:41:52.291 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2023-01-24T14:41:52.295 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2023-01-24T14:41:52.416 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2023-01-24T14:41:52.625 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:52.961 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:18.0.0-1954.g1a671c19.el8.x86_64 1/1 2023-01-24T14:41:53.455 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:53.456 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2023-01-24T14:41:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:18.0.0-1954.g1a671c19.el8.x86_64 2023-01-24T14:41:53.456 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:53.456 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2023-01-24T14:41:53.688 DEBUG:teuthology.parallel:result is None 2023-01-24T14:41:53.689 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2023-01-24T14:41:53.726 INFO:teuthology.orchestra.run.smithi105.stdout:CentOSStream 2023-01-24T14:41:53.727 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-24T14:41:53.728 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=crimson&distros=centos%2F8%2Fx86_64&sha1=1a671c198559f54389cd5afa22085a0c7b29a1ef 2023-01-24T14:41:53.881 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T14:41:53.904 INFO:teuthology.orchestra.run.smithi105.stdout:18.0.0-1954.g1a671c19.el8 2023-01-24T14:41:53.905 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1954.g1a671c19.el8 2023-01-24T14:41:53.905 INFO:teuthology.task.install:The correct ceph version 18.0.0-1954.g1a671c19 is installed. 2023-01-24T14:41:53.905 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T14:41:53.906 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:53.906 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T14:41:53.972 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T14:41:53.972 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:53.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T14:41:54.039 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T14:41:54.106 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T14:41:54.106 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:54.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T14:41:54.176 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T14:41:54.243 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T14:41:54.307 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T14:41:54.308 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2023-01-24T14:41:54.335 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T14:41:54.336 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T14:41:54.402 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T14:41:54.403 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T14:41:54.477 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T14:41:54.478 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}, 'global': {'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(POOL_FULL\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-24T14:41:54.478 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311', 'branch': 'main', 'description': 'crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format}', 'email': None, 'first_in_suite': False, 'job_id': '7135311', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(POOL_FULL\\)'], 'sha1': '1a671c198559f54389cd5afa22085a0c7b29a1ef'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'crimson', 'sha1': '1a671c198559f54389cd5afa22085a0c7b29a1ef'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '510284b66513490445619d1430aa869868c71a09'}}, 'owner': 'scheduled_rzarzynski@teuthology', 'priority': 60, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'sha1': '1a671c198559f54389cd5afa22085a0c7b29a1ef', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '510284b66513490445619d1430aa869868c71a09', 'targets': {'smithi105.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjmRpV2nHf7zNb3iCW6xG4he1DmKPZ5qBdD8qr1YwZ1S7uwiCJ5kV1hlufB/jJbinbEvAiq3NmQjHfUZpWVJvEvQUguaoqr2pag64PI5VS3FSCB4nFiCXD5zqYASyxVG1EcUSBqJBi8Hh3DQoumOVbT6J1di3pqgo13JsATpZLBd342M1EoYyW1ia/0D5llnKznvJq996rtGm2RXZ50u+1VxpteD3Pf/tZuIe2hgJThto6ZZsoCBtNf0MuIWY7hqJpltje6lhzMCkmK9zYkccrYrRtDj2LORcXaWrv6Hw6a0sR8fAHQ/v7Tb9lyTH6wFAcCRom2xN9GuWhqyVJkJRaudordVVaj2tTKFZIQ2/ax4K59QnSK9KCyS/+nBg+tVxQTdBYIKA5rNii8xymJZ8iB+nBlpmBVgTA/ilanqp95QtsMYb/Na68E0nnm/1o5nkb9E3/oQPtN8xPGmVvmomyvtN667o9Nnzas3jR0CV/L1OGwmF0Ba96de4Bti0Qu8k='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}, 'global': {'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'is full \\(reached quota', '\\(POOL_FULL\\)'], 'sha1': '1a671c198559f54389cd5afa22085a0c7b29a1ef', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rbd/test_librbd.sh']}}}], 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_14:08:39', 'tube': 'smithi', 'user': 'rzarzynski', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T14:41:54.479 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T14:41:54.528 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T14:41:54.595 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:54.596 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T14:41:54.653 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T14:41:54.654 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2023-01-24T14:41:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T14:41:54.711 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T14:41:54.712 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 368545 Links: 1 2023-01-24T14:41:54.712 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T14:41:54.712 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T14:41:54.712 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 14:41:49.045406203 +0000 2023-01-24T14:41:54.713 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 14:25:14.927122739 +0000 2023-01-24T14:41:54.713 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 14:25:14.927122739 +0000 2023-01-24T14:41:54.713 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T14:41:54.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T14:41:54.777 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T14:41:54.777 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T14:41:54.777 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000243925 s, 2.1 MB/s 2023-01-24T14:41:54.779 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T14:41:54.837 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2023-01-24T14:41:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T14:41:54.895 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T14:41:54.896 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 373817 Links: 1 2023-01-24T14:41:54.896 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T14:41:54.896 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T14:41:54.896 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 14:41:49.045406203 +0000 2023-01-24T14:41:54.897 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 14:25:15.439113109 +0000 2023-01-24T14:41:54.897 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 14:25:15.439113109 +0000 2023-01-24T14:41:54.897 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T14:41:54.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T14:41:54.961 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T14:41:54.962 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T14:41:54.962 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000229132 s, 2.2 MB/s 2023-01-24T14:41:54.963 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T14:41:55.022 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2023-01-24T14:41:55.080 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T14:41:55.081 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T14:41:55.081 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 369654 Links: 1 2023-01-24T14:41:55.081 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T14:41:55.081 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T14:41:55.081 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 14:41:49.045406203 +0000 2023-01-24T14:41:55.082 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 14:25:15.938103722 +0000 2023-01-24T14:41:55.082 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 14:25:15.938103722 +0000 2023-01-24T14:41:55.082 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T14:41:55.082 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T14:41:55.146 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T14:41:55.146 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T14:41:55.147 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000205032 s, 2.5 MB/s 2023-01-24T14:41:55.148 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T14:41:55.206 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2023-01-24T14:41:55.264 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T14:41:55.264 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T14:41:55.264 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 378189 Links: 1 2023-01-24T14:41:55.265 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T14:41:55.265 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T14:41:55.265 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2023-01-24 14:41:49.045406203 +0000 2023-01-24T14:41:55.265 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2023-01-24 14:25:16.460093903 +0000 2023-01-24T14:41:55.265 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2023-01-24 14:25:16.460093903 +0000 2023-01-24T14:41:55.266 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2023-01-24T14:41:55.266 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T14:41:55.330 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2023-01-24T14:41:55.330 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2023-01-24T14:41:55.331 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000200274 s, 2.6 MB/s 2023-01-24T14:41:55.332 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T14:41:55.391 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-24T14:41:55.391 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-01-24T14:41:55.391 INFO:tasks.ceph:Generating config... 2023-01-24T14:41:55.393 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2023-01-24T14:41:55.393 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2023-01-24T14:41:55.393 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2023-01-24T14:41:55.394 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2023-01-24T14:41:55.394 INFO:tasks.ceph:[mon] paxos service trim min = 10 2023-01-24T14:41:55.394 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T14:41:55.395 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T14:41:55.395 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T14:41:55.395 INFO:tasks.ceph:[osd] debug monc = 20 2023-01-24T14:41:55.395 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T14:41:55.396 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T14:41:55.396 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-01-24T14:41:55.396 INFO:tasks.ceph:[global] ms client mode = crc 2023-01-24T14:41:55.396 INFO:tasks.ceph:[global] ms cluster mode = crc 2023-01-24T14:41:55.397 INFO:tasks.ceph:[global] ms mon client mode = crc 2023-01-24T14:41:55.397 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2023-01-24T14:41:55.397 INFO:tasks.ceph:[global] ms mon service mode = crc 2023-01-24T14:41:55.397 INFO:tasks.ceph:[global] ms service mode = crc 2023-01-24T14:41:55.398 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T14:41:55.398 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T14:41:55.398 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T14:41:55.399 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T14:41:55.489 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T14:41:55.491 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T14:41:55.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T14:41:55.572 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105')] 2023-01-24T14:41:55.573 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2023-01-24T14:41:55.573 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:55.574 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T14:41:55.630 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T14:41:55.720 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T14:41:55.720 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:41:55.720 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = pacific 2023-01-24T14:41:55.721 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2023-01-24T14:41:55.721 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:41:55.721 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2023-01-24T14:41:55.719620+0000 2023-01-24T14:41:55.721 INFO:teuthology.orchestra.run.smithi105.stdout:created 2023-01-24T14:41:55.719620+0000 2023-01-24T14:41:55.721 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 16 (pacific) 2023-01-24T14:41:55.722 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 1 2023-01-24T14:41:55.722 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2023-01-24T14:41:55.722 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-01-24T14:41:55.723 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T14:41:55.745 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6b13f761-45e0-46c8-9934-bb4919ac75a7... 2023-01-24T14:41:55.746 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T14:41:55.831 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2023-01-24T14:41:55.831 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2023-01-24T14:41:55.831 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T14:41:55.832 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2023-01-24T14:41:55.832 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.832 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2023-01-24T14:41:55.832 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.832 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2023-01-24T14:41:55.833 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.833 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2023-01-24T14:41:55.833 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2023-01-24T14:41:55.833 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.834 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2023-01-24T14:41:55.834 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2023-01-24T14:41:55.834 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.834 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T14:41:55.834 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2023-01-24T14:41:55.834 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.835 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T14:41:55.835 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T14:41:55.835 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.836 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2023-01-24T14:41:55.836 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.836 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2023-01-24T14:41:55.836 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2023-01-24T14:41:55.837 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2023-01-24T14:41:55.837 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2023-01-24T14:41:55.837 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2023-01-24T14:41:55.837 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2023-01-24T14:41:55.837 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2023-01-24T14:41:55.837 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T14:41:55.838 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T14:41:55.838 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2023-01-24T14:41:55.838 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.838 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T14:41:55.839 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.839 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2023-01-24T14:41:55.839 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.839 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2023-01-24T14:41:55.839 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.840 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2023-01-24T14:41:55.840 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2023-01-24T14:41:55.840 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2023-01-24T14:41:55.840 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = 172.21.15.105 2023-01-24T14:41:55.841 INFO:teuthology.orchestra.run.smithi105.stdout: ms client mode = crc 2023-01-24T14:41:55.841 INFO:teuthology.orchestra.run.smithi105.stdout: ms cluster mode = crc 2023-01-24T14:41:55.841 INFO:teuthology.orchestra.run.smithi105.stdout: ms mon client mode = crc 2023-01-24T14:41:55.841 INFO:teuthology.orchestra.run.smithi105.stdout: ms mon cluster mode = crc 2023-01-24T14:41:55.841 INFO:teuthology.orchestra.run.smithi105.stdout: ms mon service mode = crc 2023-01-24T14:41:55.842 INFO:teuthology.orchestra.run.smithi105.stdout: ms service mode = crc 2023-01-24T14:41:55.842 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:41:55.842 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.842 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2023-01-24T14:41:55.842 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2023-01-24T14:41:55.843 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.843 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2023-01-24T14:41:55.843 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2023-01-24T14:41:55.843 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.843 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2023-01-24T14:41:55.844 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2023-01-24T14:41:55.844 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.844 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2023-01-24T14:41:55.844 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2023-01-24T14:41:55.845 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2023-01-24T14:41:55.845 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.845 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2023-01-24T14:41:55.845 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2023-01-24T14:41:55.845 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.846 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2023-01-24T14:41:55.846 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.846 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2023-01-24T14:41:55.846 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.846 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2023-01-24T14:41:55.846 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.847 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2023-01-24T14:41:55.847 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2023-01-24T14:41:55.847 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.847 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2023-01-24T14:41:55.848 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2023-01-24T14:41:55.848 INFO:teuthology.orchestra.run.smithi105.stdout: debug monc = 20 2023-01-24T14:41:55.848 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-01-24T14:41:55.848 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2023-01-24T14:41:55.848 INFO:teuthology.orchestra.run.smithi105.stdout: osd shutdown pgref assert = True 2023-01-24T14:41:55.849 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.849 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2023-01-24T14:41:55.849 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-01-24T14:41:55.849 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2023-01-24T14:41:55.849 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-01-24T14:41:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-01-24T14:41:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-01-24T14:41:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-01-24T14:41:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2023-01-24T14:41:55.850 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.851 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2023-01-24T14:41:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2023-01-24T14:41:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2023-01-24T14:41:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2023-01-24T14:41:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2023-01-24T14:41:55.851 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2023-01-24T14:41:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2023-01-24T14:41:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2023-01-24T14:41:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2023-01-24T14:41:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2023-01-24T14:41:55.852 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2023-01-24T14:41:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T14:41:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T14:41:55.853 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T14:41:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T14:41:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T14:41:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T14:41:55.854 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2023-01-24T14:41:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2023-01-24T14:41:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.855 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2023-01-24T14:41:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: mon min osdmap epochs = 50 2023-01-24T14:41:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune interval = 2 2023-01-24T14:41:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune min = 15 2023-01-24T14:41:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: mon osdmap full prune txsize = 2 2023-01-24T14:41:55.856 INFO:teuthology.orchestra.run.smithi105.stdout: paxos service trim min = 10 2023-01-24T14:41:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:41:55.857 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2023-01-24T14:41:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2023-01-24T14:41:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2023-01-24T14:41:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2023-01-24T14:41:55.857 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T14:41:55.858 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T14:41:55.858 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2023-01-24T14:41:55.859 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T14:41:55.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T14:41:55.966 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T14:41:55.966 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:55.967 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T14:41:55.985 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:55.985 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T14:41:56.043 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T14:41:56.044 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:56.044 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T14:41:56.044 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T14:41:56.120 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:41:56.120 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T14:41:56.178 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T14:41:56.179 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T14:41:56.179 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T14:41:56.279 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T14:41:56.281 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T14:41:56.282 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T14:41:56.282 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T14:41:56.333 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T14:41:56.345 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T14:41:56.346 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-01-24T14:41:56.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T14:41:56.373 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-24T14:41:56.373 INFO:tasks.ceph:role: osd.0 2023-01-24T14:41:56.374 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T14:41:56.374 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T14:41:56.448 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T14:41:56.448 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T14:41:56.448 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T14:41:56.448 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T14:41:56.449 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T14:41:56.449 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-01-24T14:41:56.449 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T14:41:56.449 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T14:41:56.449 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T14:41:56.450 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T14:41:56.884 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-01-24T14:41:56.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-01-24T14:41:56.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T14:41:57.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T14:41:57.095 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T14:41:57.162 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-24T14:41:57.163 INFO:tasks.ceph:role: osd.1 2023-01-24T14:41:57.163 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T14:41:57.163 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T14:41:57.237 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T14:41:57.237 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T14:41:57.237 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T14:41:57.237 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T14:41:57.238 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T14:41:57.238 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-01-24T14:41:57.238 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T14:41:57.238 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T14:41:57.238 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T14:41:57.239 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T14:41:57.655 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-01-24T14:41:57.657 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-01-24T14:41:57.658 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T14:41:57.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T14:41:57.790 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T14:41:57.858 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-01-24T14:41:57.858 INFO:tasks.ceph:role: osd.2 2023-01-24T14:41:57.858 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T14:41:57.859 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T14:41:57.933 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T14:41:57.934 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T14:41:57.934 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T14:41:57.935 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-24T14:41:57.935 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T14:41:57.935 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2023-01-24T14:41:57.936 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T14:41:57.937 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T14:41:57.937 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T14:41:57.937 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T14:41:58.372 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2023-01-24T14:41:58.374 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2023-01-24T14:41:58.374 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T14:41:58.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T14:41:58.577 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T14:41:59.364 INFO:teuthology.orchestra.run.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:41:59.365 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,364 seastar - Reactor backend: epoll 2023-01-24T14:41:59.373 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,372 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:41:59.373 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,372 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:41:59.373 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,372 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:41:59.374 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,373 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:41:59.375 INFO:teuthology.orchestra.run.smithi105.stderr:==100508==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:41:59.382 INFO:teuthology.orchestra.run.smithi105.stderr:==100508==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffcb1d35000; bottom 0x7f4b7c71c000; size: 0x00b135619000 (761104797696) 2023-01-24T14:41:59.382 INFO:teuthology.orchestra.run.smithi105.stderr:False positive error reports may follow 2023-01-24T14:41:59.382 INFO:teuthology.orchestra.run.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:41:59.417 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,416 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-01-24T14:41:59.418 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,416 [shard 0] osd - get_early_config: set --smp 1 2023-01-24T14:41:59.896 INFO:teuthology.orchestra.run.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:41:59.897 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,896 seastar - Reactor backend: epoll 2023-01-24T14:41:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,897 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:41:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,898 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:41:59.898 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,898 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:41:59.899 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:41:59,898 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:41:59.900 INFO:teuthology.orchestra.run.smithi105.stderr:==100506==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:41:59.906 INFO:teuthology.orchestra.run.smithi105.stderr:==100506==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffcb1d35000; bottom 0x7f4b7c715000; size: 0x00b135620000 (761104826368) 2023-01-24T14:41:59.906 INFO:teuthology.orchestra.run.smithi105.stderr:False positive error reports may follow 2023-01-24T14:41:59.906 INFO:teuthology.orchestra.run.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:02.830 INFO:teuthology.orchestra.run.smithi105.stdout:created object store /var/lib/ceph/osd/ceph-0 for osd.0 fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:42:02.889 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T14:42:02.917 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T14:42:03.689 INFO:teuthology.orchestra.run.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:03.690 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:03,690 seastar - Reactor backend: epoll 2023-01-24T14:42:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:03,698 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:03,698 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:03.699 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:03,698 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:03.700 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:03,699 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:03.701 INFO:teuthology.orchestra.run.smithi105.stderr:==101326==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:03.707 INFO:teuthology.orchestra.run.smithi105.stderr:==101326==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe89ead000; bottom 0x7f109d91d000; size: 0x00edec590000 (1021872504832) 2023-01-24T14:42:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:03.708 INFO:teuthology.orchestra.run.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:03,742 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-01-24T14:42:03.743 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:03,742 [shard 0] osd - get_early_config: set --smp 1 2023-01-24T14:42:04.223 INFO:teuthology.orchestra.run.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:04.223 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:04,223 seastar - Reactor backend: epoll 2023-01-24T14:42:04.225 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:04,224 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:04.225 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:04,225 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:04.225 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:04,225 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:04.226 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:04,225 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:04.227 INFO:teuthology.orchestra.run.smithi105.stderr:==101324==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:04.233 INFO:teuthology.orchestra.run.smithi105.stderr:==101324==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe89ead000; bottom 0x7f109d916000; size: 0x00edec597000 (1021872533504) 2023-01-24T14:42:04.233 INFO:teuthology.orchestra.run.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:04.233 INFO:teuthology.orchestra.run.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:07.136 INFO:teuthology.orchestra.run.smithi105.stdout:created object store /var/lib/ceph/osd/ceph-1 for osd.1 fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:42:07.220 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T14:42:07.248 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T14:42:08.016 INFO:teuthology.orchestra.run.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,017 seastar - Reactor backend: epoll 2023-01-24T14:42:08.025 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,024 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,025 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,025 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:08.026 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,025 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:08.027 INFO:teuthology.orchestra.run.smithi105.stderr:==102144==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:08.034 INFO:teuthology.orchestra.run.smithi105.stderr:==102144==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffcfb86c000; bottom 0x7fd0f7dd9000; size: 0x002c03a93000 (189039980544) 2023-01-24T14:42:08.034 INFO:teuthology.orchestra.run.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:08.034 INFO:teuthology.orchestra.run.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:08.069 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,068 [shard 0] osd - bypassing the config fetch due to --no-mon-config 2023-01-24T14:42:08.069 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,068 [shard 0] osd - get_early_config: set --smp 1 2023-01-24T14:42:08.554 INFO:teuthology.orchestra.run.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:08.554 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,554 seastar - Reactor backend: epoll 2023-01-24T14:42:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,555 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:08.556 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,556 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,556 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:08.557 INFO:teuthology.orchestra.run.smithi105.stderr:INFO 2023-01-24 14:42:08,556 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:08.558 INFO:teuthology.orchestra.run.smithi105.stderr:==102142==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:08.564 INFO:teuthology.orchestra.run.smithi105.stderr:==102142==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffcfb86c000; bottom 0x7fd0f7dd2000; size: 0x002c03a9a000 (189040009216) 2023-01-24T14:42:08.564 INFO:teuthology.orchestra.run.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:08.564 INFO:teuthology.orchestra.run.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:11.480 INFO:teuthology.orchestra.run.smithi105.stdout:created object store /var/lib/ceph/osd/ceph-2 for osd.2 fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:42:11.537 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T14:42:11.566 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T14:42:11.566 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:11.567 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T14:42:11.633 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:11.634 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T14:42:11.700 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:11.701 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T14:42:11.767 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:11.768 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T14:42:11.835 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:11.835 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T14:42:11.893 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T14:42:11.894 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T14:42:11.959 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2023-01-24T14:42:11.959 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA07s9jyVGeEBAAK8TGb9ZBPCSuFiDxh9BI8g== 2023-01-24T14:42:11.960 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2023-01-24T14:42:11.960 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA37s9jw4seORAAEwve3lcx78aWA+DxXF40Qg== 2023-01-24T14:42:11.960 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2023-01-24T14:42:11.960 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA87s9jHFbVEBAAr3uWqzsfzpGPLjrwisdL7g== 2023-01-24T14:42:11.960 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2023-01-24T14:42:11.961 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBA7s9jm72JJBAA7sWz93L1kFlAOEZzLBkduw== 2023-01-24T14:42:11.961 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2023-01-24T14:42:11.961 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA07s9j6p7VExAAxUCCD1VDGOPQHY7aenURLg== 2023-01-24T14:42:11.962 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T14:42:12.064 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T14:42:12.125 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T14:42:12.186 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T14:42:12.248 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T14:42:12.312 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T14:42:12.313 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T14:42:12.341 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T14:42:12.457 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T14:42:12.484 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T14:42:12.542 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T14:42:12.543 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T14:42:12.544 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T14:42:12.585 INFO:tasks.ceph.mon.a:Started 2023-01-24T14:42:12.585 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T14:42:12.586 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T14:42:12.586 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T14:42:12.588 INFO:tasks.ceph.mgr.x:Started 2023-01-24T14:42:12.589 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T14:42:12.589 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T14:42:12.818 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2023-01-24T14:42:12.829 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T14:42:12.829 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:12.830 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T14:42:12.859 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:12.859 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T14:42:12.927 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:42:12.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T14:42:12.994 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 96902593-f039-4d73-b201-75d8f0bf791a 0 2023-01-24T14:42:13.256 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T14:42:13.269 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new 9dc43bde-f8d5-45c7-a5a2-5a32399712cf 1 2023-01-24T14:42:13.512 INFO:teuthology.orchestra.run.smithi105.stdout:1 2023-01-24T14:42:13.525 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd new e1f033bf-7672-49c7-8caa-57461f6e67b7 2 2023-01-24T14:42:13.745 INFO:teuthology.orchestra.run.smithi105.stdout:2 2023-01-24T14:42:13.758 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T14:42:13.758 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T14:42:13.760 INFO:tasks.ceph.osd.0:Started 2023-01-24T14:42:13.761 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T14:42:13.762 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T14:42:13.764 INFO:tasks.ceph.osd.1:Started 2023-01-24T14:42:13.764 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T14:42:13.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T14:42:13.767 INFO:tasks.ceph.osd.2:Started 2023-01-24T14:42:13.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T14:42:13.942 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:13.940+0000 7fb816a5d1c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T14:42:14.037 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:14.036+0000 7fb816a5d1c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T14:42:14.138 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:14.137+0000 7fb816a5d1c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T14:42:14.157 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:42:14.157 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":5,"fsid":"6b13f761-45e0-46c8-9934-bb4919ac75a7","created":"2023-01-24T14:42:12.726827+0000","modified":"2023-01-24T14:42:13.742705+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T14:42:13.742705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"96902593-f039-4d73-b201-75d8f0bf791a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9dc43bde-f8d5-45c7-a5a2-5a32399712cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e1f033bf-7672-49c7-8caa-57461f6e67b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T14:42:14.192 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T14:42:14.193 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T14:42:14.374 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:14.372+0000 7fb816a5d1c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T14:42:14.495 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T14:42:14.679 INFO:tasks.ceph.osd.2.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:14.679 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,679 seastar - Reactor backend: epoll 2023-01-24T14:42:14.687 INFO:tasks.ceph.osd.0.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:14.688 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,687 seastar - Reactor backend: epoll 2023-01-24T14:42:14.695 INFO:tasks.ceph.osd.1.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:14.696 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,695 seastar - Reactor backend: epoll 2023-01-24T14:42:14.754 INFO:teuthology.misc.health.smithi105.stdout: 2023-01-24T14:42:14.754 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":5,"fsid":"6b13f761-45e0-46c8-9934-bb4919ac75a7","created":"2023-01-24T14:42:12.726827+0000","modified":"2023-01-24T14:42:13.742705+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T14:42:13.742705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"96902593-f039-4d73-b201-75d8f0bf791a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9dc43bde-f8d5-45c7-a5a2-5a32399712cf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e1f033bf-7672-49c7-8caa-57461f6e67b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T14:42:14.765 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-01-24T14:42:14.779 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,778 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:14.779 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,778 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:14.779 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,778 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:14.780 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,778 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:14.781 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,781 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:14.781 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,781 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:14.782 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,781 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:14.782 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,781 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:14.784 INFO:tasks.ceph.osd.0.smithi105.stderr:==103876==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:14.784 INFO:tasks.ceph.osd.1.smithi105.stderr:==103877==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:14.797 INFO:tasks.ceph.osd.1.smithi105.stderr:==103877==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc11fd9000; bottom 0x7f3f38dea000; size: 0x00bcd91ef000 (811096535040) 2023-01-24T14:42:14.797 INFO:tasks.ceph.osd.1.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:14.797 INFO:tasks.ceph.osd.1.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:14.798 INFO:tasks.ceph.osd.0.smithi105.stderr:==103876==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe78d8b000; bottom 0x7fd5ac99b000; size: 0x0028cc3f0000 (175225372672) 2023-01-24T14:42:14.798 INFO:tasks.ceph.osd.0.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:14.798 INFO:tasks.ceph.osd.0.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:14.809 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,809 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:14.810 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,809 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:14.810 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,809 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:14.810 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,809 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:14.812 INFO:tasks.ceph.osd.2.smithi105.stderr:==103875==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:14.818 INFO:tasks.ceph.osd.2.smithi105.stderr:==103875==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fffd30f4000; bottom 0x7f6e64813000; size: 0x00916e8e1000 (624625061888) 2023-01-24T14:42:14.819 INFO:tasks.ceph.osd.2.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:14.819 INFO:tasks.ceph.osd.2.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:14.899 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,899 [shard 0] osd - populating config from monitor 2023-01-24T14:42:14.900 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,899 [shard 0] osd - populating config from monitor 2023-01-24T14:42:14.900 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] osd - populating config from monitor 2023-01-24T14:42:14.902 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding auth protocol: cephx 2023-01-24T14:42:14.902 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding auth protocol: cephx 2023-01-24T14:42:14.902 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding auth protocol: cephx 2023-01-24T14:42:14.903 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding con mode: crc 2023-01-24T14:42:14.903 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding con mode: crc 2023-01-24T14:42:14.903 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding con mode: crc 2023-01-24T14:42:14.903 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding con mode: crc 2023-01-24T14:42:14.903 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,900 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding con mode: crc 2023-01-24T14:42:14.904 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding auth protocol: cephx 2023-01-24T14:42:14.905 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding auth protocol: cephx 2023-01-24T14:42:14.905 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding auth protocol: cephx 2023-01-24T14:42:14.905 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding con mode: crc 2023-01-24T14:42:14.905 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding con mode: crc 2023-01-24T14:42:14.905 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding con mode: crc 2023-01-24T14:42:14.906 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding con mode: crc 2023-01-24T14:42:14.906 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding con mode: crc 2023-01-24T14:42:14.906 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f3f38dae6b0) adding con mode: crc 2023-01-24T14:42:14.907 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding auth protocol: cephx 2023-01-24T14:42:14.907 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding auth protocol: cephx 2023-01-24T14:42:14.907 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding auth protocol: cephx 2023-01-24T14:42:14.907 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding con mode: crc 2023-01-24T14:42:14.908 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding con mode: crc 2023-01-24T14:42:14.908 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7f6e647d76b0) adding con mode: crc 2023-01-24T14:42:14.908 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,902 [shard 0] auth - auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T14:42:14.909 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,902 [shard 0] auth - auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T14:42:14.910 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,901 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding con mode: crc 2023-01-24T14:42:14.910 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,902 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding con mode: crc 2023-01-24T14:42:14.910 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,902 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding con mode: crc 2023-01-24T14:42:14.910 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,903 [shard 0] auth - AuthRegistry(0x7fd5ac95f6b0) adding con mode: crc 2023-01-24T14:42:14.910 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,903 [shard 0] auth - auth: KeyRing::load: loaded key file /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T14:42:14.918 INFO:tasks.ceph.osd.2.smithi105.stderr:DEBUG 2023-01-24 14:42:14,918 [shard 0] monc - build_monmap: for_mkfs=false 2023-01-24T14:42:14.919 INFO:tasks.ceph.osd.0.smithi105.stderr:DEBUG 2023-01-24 14:42:14,918 [shard 0] monc - build_monmap: for_mkfs=false 2023-01-24T14:42:14.919 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,918 [shard 0] monc - reopen_session to mon.-1 2023-01-24T14:42:14.920 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,918 [shard 0] monc - connecting to mon.1 2023-01-24T14:42:14.920 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,918 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) - >> mon.? v2:172.21.15.105:3300/0] ProtocolV2::start_connect(): peer_addr=v2:172.21.15.105:3300/0, peer_name=mon.?, cc=1701941779321803313 policy(lossy=false, server=false, standby=false, resetcheck=true) 2023-01-24T14:42:14.921 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,919 [shard 0] monc - reopen_session to mon.-1 2023-01-24T14:42:14.921 INFO:tasks.ceph.osd.2.smithi105.stderr:WARN 2023-01-24 14:42:14,919 [shard 0] monc - mon.0 does not have an addr compatible with me 2023-01-24T14:42:14.921 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,919 [shard 0] monc - connecting to mon.1 2023-01-24T14:42:14.921 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,919 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) - >> mon.? v2:172.21.15.105:3300/0] ProtocolV2::start_connect(): peer_addr=v2:172.21.15.105:3300/0, peer_name=mon.?, cc=15786297194304456464 policy(lossy=false, server=false, standby=false, resetcheck=true) 2023-01-24T14:42:14.922 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,919 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) - >> mon.? v2:172.21.15.105:3300/0] write_event: delay ... 2023-01-24T14:42:14.922 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,919 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) - >> mon.? v2:172.21.15.105:3300/0] write_event: delay ... 2023-01-24T14:42:14.922 INFO:tasks.ceph.osd.0.smithi105.stderr:WARN 2023-01-24 14:42:14,919 [shard 0] monc - mon.0 does not have an addr compatible with me 2023-01-24T14:42:14.923 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) -@60488 >> mon.? v2:172.21.15.105:3300/0] peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60488/0 (socket says 172.21.15.105:60488) 2023-01-24T14:42:14.923 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.? v2:172.21.15.105:3300/0] learned myaddr=172.21.15.105:0/103875 (unbound) 2023-01-24T14:42:14.924 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] monc - get_auth_request(conn=[0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.? v2:172.21.15.105:3300/0], auth_method=0) 2023-01-24T14:42:14.924 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] monc - method 2 preferred_modes [1] 2023-01-24T14:42:14.924 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) -@65285 >> mon.? v2:172.21.15.105:3300/0] peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:65285/0 (socket says 172.21.15.105:65285) 2023-01-24T14:42:14.925 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.? v2:172.21.15.105:3300/0] learned myaddr=172.21.15.105:0/103876 (unbound) 2023-01-24T14:42:14.925 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] monc - get_auth_request(conn=[0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.? v2:172.21.15.105:3300/0], auth_method=0) 2023-01-24T14:42:14.925 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,921 [shard 0] monc - method 2 preferred_modes [1] 2023-01-24T14:42:14.926 INFO:tasks.ceph.osd.1.smithi105.stderr:DEBUG 2023-01-24 14:42:14,922 [shard 0] monc - build_monmap: for_mkfs=false 2023-01-24T14:42:14.926 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,923 [shard 0] monc - reopen_session to mon.-1 2023-01-24T14:42:14.927 INFO:tasks.ceph.osd.1.smithi105.stderr:WARN 2023-01-24 14:42:14,923 [shard 0] monc - mon.0 does not have an addr compatible with me 2023-01-24T14:42:14.927 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,923 [shard 0] monc - connecting to mon.1 2023-01-24T14:42:14.927 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,923 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) - >> mon.? v2:172.21.15.105:3300/0] ProtocolV2::start_connect(): peer_addr=v2:172.21.15.105:3300/0, peer_name=mon.?, cc=10638447535936742671 policy(lossy=false, server=false, standby=false, resetcheck=true) 2023-01-24T14:42:14.927 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,923 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) - >> mon.? v2:172.21.15.105:3300/0] write_event: delay ... 2023-01-24T14:42:14.927 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,924 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) -@58978 >> mon.? v2:172.21.15.105:3300/0] peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:58978/0 (socket says 172.21.15.105:58978) 2023-01-24T14:42:14.928 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,924 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.? v2:172.21.15.105:3300/0] learned myaddr=172.21.15.105:0/103877 (unbound) 2023-01-24T14:42:14.928 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,924 [shard 0] monc - get_auth_request(conn=[0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.? v2:172.21.15.105:3300/0], auth_method=0) 2023-01-24T14:42:14.928 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,924 [shard 0] monc - method 2 preferred_modes [1] 2023-01-24T14:42:14.962 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,961 [shard 0] monc - responding with 36 bytes 2023-01-24T14:42:14.962 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,962 [shard 0] monc - found mon.noname-a 2023-01-24T14:42:14.963 INFO:tasks.ceph.osd.0.smithi105.stderr:DEBUG 2023-01-24 14:42:14,962 [shard 0] monc - reopen_session mon connection attempts complete 2023-01-24T14:42:14.963 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,962 [shard 0] monc - responding with 36 bytes 2023-01-24T14:42:14.964 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,963 [shard 0] monc - found mon.noname-a 2023-01-24T14:42:14.964 INFO:tasks.ceph.osd.2.smithi105.stderr:DEBUG 2023-01-24 14:42:14,963 [shard 0] monc - reopen_session mon connection attempts complete 2023-01-24T14:42:14.964 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,963 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571304.9633236) 2023-01-24T14:42:14.964 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,963 [shard 0] monc - sending auth(proto 2 2 bytes epoch 0) v1 2023-01-24T14:42:14.965 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:14,963 [shard 0] monc - waiting 2023-01-24T14:42:14.965 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,963 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571304.9639466) 2023-01-24T14:42:14.965 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,963 [shard 0] monc - sending auth(proto 2 2 bytes epoch 0) v1 2023-01-24T14:42:14.966 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:14,964 [shard 0] monc - waiting 2023-01-24T14:42:14.966 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,964 [shard 0] monc - responding with 36 bytes 2023-01-24T14:42:14.967 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,965 [shard 0] monc - found mon.noname-a 2023-01-24T14:42:14.967 INFO:tasks.ceph.osd.1.smithi105.stderr:DEBUG 2023-01-24 14:42:14,965 [shard 0] monc - reopen_session mon connection attempts complete 2023-01-24T14:42:14.967 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,965 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571304.9655774) 2023-01-24T14:42:14.967 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,965 [shard 0] monc - sending auth(proto 2 2 bytes epoch 0) v1 2023-01-24T14:42:14.967 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:14,965 [shard 0] monc - waiting 2023-01-24T14:42:15.005 INFO:tasks.ceph.osd.2.smithi105.stderr:WARN 2023-01-24 14:42:15,004 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.0 v2:172.21.15.105:3300/0] UPDATE Policy(lossy=true) from server flags 2023-01-24T14:42:15.005 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,004 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.0 v2:172.21.15.105:3300/0] connected: gs=1, pgs=22, cs=0, client_cookie=15786297194304456464, server_cookie=0, in_seq=0, out_seq=0, out_q=2 2023-01-24T14:42:15.006 INFO:tasks.ceph.osd.0.smithi105.stderr:WARN 2023-01-24 14:42:15,004 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.0 v2:172.21.15.105:3300/0] UPDATE Policy(lossy=true) from server flags 2023-01-24T14:42:15.006 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,005 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.0 v2:172.21.15.105:3300/0] connected: gs=1, pgs=23, cs=0, client_cookie=1701941779321803313, server_cookie=0, in_seq=0, out_seq=0, out_q=2 2023-01-24T14:42:15.007 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - got monmap 1, mon.a, is now rank 0 2023-01-24T14:42:15.007 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - handle_monmap: renewing tickets 2023-01-24T14:42:15.007 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571305.0061078) 2023-01-24T14:42:15.007 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - renew_rotating_keyring called too often (last: 1674571334.9639466) 2023-01-24T14:42:15.009 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - got monmap 1, mon.a, is now rank 0 2023-01-24T14:42:15.009 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - handle_monmap: renewing tickets 2023-01-24T14:42:15.010 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571305.005701) 2023-01-24T14:42:15.010 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - renew_rotating_keyring called too often (last: 1674571334.9633236) 2023-01-24T14:42:15.010 INFO:tasks.ceph.osd.2.smithi105.stderr:DEBUG 2023-01-24 14:42:15,006 [shard 0] monc - renew_tickets: don't need new tickets 2023-01-24T14:42:15.010 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - handle_mon_map: renewed tickets 2023-01-24T14:42:15.010 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - handle_auth_reply [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.0 v2:172.21.15.105:3300/0] returns auth_reply(proto 2 0 (0) Success) v1: 0 2023-01-24T14:42:15.011 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - handle_auth_reply 2023-01-24T14:42:15.011 INFO:tasks.ceph.osd.2.smithi105.stderr:DEBUG 2023-01-24 14:42:15,006 [shard 0] monc - renew_tickets: don't need new tickets 2023-01-24T14:42:15.011 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571305.0066779) 2023-01-24T14:42:15.011 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - renew_rotating_keyring called too often (last: 1674571334.9633236) 2023-01-24T14:42:15.011 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - do_auth_single: [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.0 v2:172.21.15.105:3300/0] returns auth_reply(proto 2 0 (0) Success) v1: 0 2023-01-24T14:42:15.012 INFO:tasks.ceph.osd.0.smithi105.stderr:DEBUG 2023-01-24 14:42:15,006 [shard 0] monc - renew_tickets: don't need new tickets 2023-01-24T14:42:15.013 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,006 [shard 0] monc - handle_mon_map: renewed tickets 2023-01-24T14:42:15.013 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,007 [shard 0] monc - handle_auth_reply [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.0 v2:172.21.15.105:3300/0] returns auth_reply(proto 2 0 (0) Success) v1: 0 2023-01-24T14:42:15.013 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,007 [shard 0] monc - handle_auth_reply 2023-01-24T14:42:15.013 INFO:tasks.ceph.osd.0.smithi105.stderr:DEBUG 2023-01-24 14:42:15,007 [shard 0] monc - renew_tickets: don't need new tickets 2023-01-24T14:42:15.013 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,007 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571305.0071044) 2023-01-24T14:42:15.014 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,007 [shard 0] monc - renew_rotating_keyring called too often (last: 1674571334.9639466) 2023-01-24T14:42:15.014 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,007 [shard 0] monc - do_auth_single: [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.0 v2:172.21.15.105:3300/0] returns auth_reply(proto 2 0 (0) Success) v1: 0 2023-01-24T14:42:15.014 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,008 [shard 0] monc - set_mon_vals no callback set 2023-01-24T14:42:15.015 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,008 [shard 0] monc - set_mon_vals no callback set 2023-01-24T14:42:15.015 INFO:tasks.ceph.osd.2.smithi105.stderr:DEBUG 2023-01-24 14:42:15,014 [shard 0] osd - operator(): got config from monitor, fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:42:15.015 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,014 [shard 0] monc - stop 2023-01-24T14:42:15.016 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,014 [shard 0] monc - close 2023-01-24T14:42:15.016 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,014 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.0 v2:172.21.15.105:3300/0] closing: reset no, replace no 2023-01-24T14:42:15.017 INFO:tasks.ceph.osd.0.smithi105.stderr:DEBUG 2023-01-24 14:42:15,014 [shard 0] osd - operator(): got config from monitor, fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:42:15.017 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,014 [shard 0] monc - stop 2023-01-24T14:42:15.017 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,014 [shard 0] monc - close 2023-01-24T14:42:15.017 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,014 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.0 v2:172.21.15.105:3300/0] closing: reset no, replace no 2023-01-24T14:42:15.017 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,015 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103876@65285 >> mon.0 v2:172.21.15.105:3300/0] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof) 2023-01-24T14:42:15.018 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,015 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103875@60488 >> mon.0 v2:172.21.15.105:3300/0] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof) 2023-01-24T14:42:15.018 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:15,015 [shard 0] osd - get_early_config: set --smp 1 2023-01-24T14:42:15.018 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:15,015 [shard 0] osd - get_early_config: set --smp 1 2023-01-24T14:42:15.020 INFO:tasks.ceph.osd.1.smithi105.stderr:WARN 2023-01-24 14:42:15,019 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.0 v2:172.21.15.105:3300/0] UPDATE Policy(lossy=true) from server flags 2023-01-24T14:42:15.020 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,019 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.0 v2:172.21.15.105:3300/0] connected: gs=1, pgs=24, cs=0, client_cookie=10638447535936742671, server_cookie=0, in_seq=0, out_seq=0, out_q=2 2023-01-24T14:42:15.021 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - got monmap 1, mon.a, is now rank 0 2023-01-24T14:42:15.021 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - handle_monmap: renewing tickets 2023-01-24T14:42:15.022 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571305.0203104) 2023-01-24T14:42:15.022 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - renew_rotating_keyring called too often (last: 1674571334.9655774) 2023-01-24T14:42:15.022 INFO:tasks.ceph.osd.1.smithi105.stderr:DEBUG 2023-01-24 14:42:15,020 [shard 0] monc - renew_tickets: don't need new tickets 2023-01-24T14:42:15.022 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - handle_mon_map: renewed tickets 2023-01-24T14:42:15.022 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - handle_auth_reply [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.0 v2:172.21.15.105:3300/0] returns auth_reply(proto 2 0 (0) Success) v1: 0 2023-01-24T14:42:15.023 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - handle_auth_reply 2023-01-24T14:42:15.023 INFO:tasks.ceph.osd.1.smithi105.stderr:DEBUG 2023-01-24 14:42:15,020 [shard 0] monc - renew_tickets: don't need new tickets 2023-01-24T14:42:15.023 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - renew_rotating_keyring renewing rotating keys (they expired before 1674571305.0207713) 2023-01-24T14:42:15.023 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - renew_rotating_keyring called too often (last: 1674571334.9655774) 2023-01-24T14:42:15.023 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,020 [shard 0] monc - do_auth_single: [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.0 v2:172.21.15.105:3300/0] returns auth_reply(proto 2 0 (0) Success) v1: 0 2023-01-24T14:42:15.024 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,021 [shard 0] monc - set_mon_vals no callback set 2023-01-24T14:42:15.025 INFO:tasks.ceph.osd.1.smithi105.stderr:DEBUG 2023-01-24 14:42:15,024 [shard 0] osd - operator(): got config from monitor, fsid 6b13f761-45e0-46c8-9934-bb4919ac75a7 2023-01-24T14:42:15.025 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,024 [shard 0] monc - stop 2023-01-24T14:42:15.025 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,024 [shard 0] monc - close 2023-01-24T14:42:15.025 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,024 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.0 v2:172.21.15.105:3300/0] closing: reset no, replace no 2023-01-24T14:42:15.025 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,024 [shard 0] ms - [0x614000008040 client.?(temp_mon_client) 172.21.15.105:0/103877@58978 >> mon.0 v2:172.21.15.105:3300/0] execute_ready(): protocol aborted at CLOSING -- std::system_error (error crimson::net:4, read eof) 2023-01-24T14:42:15.026 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:15,025 [shard 0] osd - get_early_config: set --smp 1 2023-01-24T14:42:15.267 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:15.265+0000 7fb816a5d1c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T14:42:15.421 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:15.419+0000 7fb816a5d1c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T14:42:15.658 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:15.656+0000 7fb816a5d1c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T14:42:15.736 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:15.735+0000 7fb816a5d1c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T14:42:15.883 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:15.882+0000 7fb816a5d1c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T14:42:16.166 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:16.165+0000 7fb816a5d1c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-24T14:42:16.362 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:16.360+0000 7fb816a5d1c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T14:42:16.461 INFO:tasks.ceph.osd.0.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:16.461 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:16,461 seastar - Reactor backend: epoll 2023-01-24T14:42:16.470 INFO:tasks.ceph.osd.2.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:16.470 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:16,470 seastar - Reactor backend: epoll 2023-01-24T14:42:16.478 INFO:tasks.ceph.osd.1.smithi105.stderr:WARNING: debug mode. Not for benchmarking or production 2023-01-24T14:42:16.479 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:16,479 seastar - Reactor backend: epoll 2023-01-24T14:42:16.509 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:16.508+0000 7fb816a5d1c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T14:42:16.560 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:16,559 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:16.560 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:16,559 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:16.560 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:16,559 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:16.561 INFO:tasks.ceph.osd.1.smithi105.stderr:INFO 2023-01-24 14:42:16,559 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:16.561 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:16,561 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:16.562 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:16,561 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:16.562 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:16,561 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:16.562 INFO:tasks.ceph.osd.2.smithi105.stderr:INFO 2023-01-24 14:42:16,561 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:16.563 INFO:tasks.ceph.osd.2.smithi105.stderr:==103822==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:16.564 INFO:tasks.ceph.osd.1.smithi105.stderr:==103821==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:16.576 INFO:tasks.ceph.osd.2.smithi105.stderr:==103822==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7fffd30f4000; bottom 0x7f6e6480c000; size: 0x00916e8e8000 (624625090560) 2023-01-24T14:42:16.576 INFO:tasks.ceph.osd.2.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:16.576 INFO:tasks.ceph.osd.2.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:16.577 INFO:tasks.ceph.osd.1.smithi105.stderr:==103821==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffc11fd9000; bottom 0x7f3f38de3000; size: 0x00bcd91f6000 (811096563712) 2023-01-24T14:42:16.577 INFO:tasks.ceph.osd.1.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:16.577 INFO:tasks.ceph.osd.1.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:16.591 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:16,590 [shard 0] seastar - Created fair group io-queue-0, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000 2023-01-24T14:42:16.592 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:16,590 [shard 0] seastar - IO queue uses 0.75ms latency goal for device 0 2023-01-24T14:42:16.592 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:16,590 [shard 0] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647 2023-01-24T14:42:16.592 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:42:16,590 [shard 0] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000 2023-01-24T14:42:16.593 INFO:tasks.ceph.osd.0.smithi105.stderr:==103820==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2023-01-24T14:42:16.599 INFO:tasks.ceph.osd.0.smithi105.stderr:==103820==WARNING: ASan is ignoring requested __asan_handle_no_return: stack type: default top: 0x7ffe78d8b000; bottom 0x7fd5ac994000; size: 0x0028cc3f7000 (175225401344) 2023-01-24T14:42:16.599 INFO:tasks.ceph.osd.0.smithi105.stderr:False positive error reports may follow 2023-01-24T14:42:16.600 INFO:tasks.ceph.osd.0.smithi105.stderr:For details see https://github.com/google/sanitizers/issues/189 2023-01-24T14:42:16.628 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:16.626+0000 7fb816a5d1c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T14:42:16.770 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:16.769+0000 7fb816a5d1c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T14:42:16.850 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:16.848+0000 7fb816a5d1c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T14:42:17.072 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:17.070+0000 7fb816a5d1c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T14:42:17.151 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:17.150+0000 7fb816a5d1c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T14:42:17.830 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:17.829+0000 7fb816a5d1c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T14:42:18.082 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:18.080+0000 7fb816a5d1c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T14:42:18.157 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:18.156+0000 7fb816a5d1c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T14:42:18.238 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:18.236+0000 7fb816a5d1c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T14:42:18.421 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:18.419+0000 7fb816a5d1c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T14:42:18.492 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:18.490+0000 7fb816a5d1c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T14:42:18.646 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:18.645+0000 7fb816a5d1c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T14:42:19.675 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:42:19.673+0000 7fb7d30dc700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-01-24T14:42:21.069 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T14:42:21.394 INFO:teuthology.misc.health.smithi105.stdout: 2023-01-24T14:42:21.394 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":9,"fsid":"6b13f761-45e0-46c8-9934-bb4919ac75a7","created":"2023-01-24T14:42:12.726827+0000","modified":"2023-01-24T14:42:20.678264+0000","last_up_change":"2023-01-24T14:42:18.666231+0000","last_in_change":"2023-01-24T14:42:13.742705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T14:42:18.692732+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"9","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"96902593-f039-4d73-b201-75d8f0bf791a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6801","nonce":103820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":103820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6809","nonce":103820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6807","nonce":103820}]},"public_addr":"172.21.15.105:6801/103820","cluster_addr":"172.21.15.105:6800/103820","heartbeat_back_addr":"172.21.15.105:6809/103820","heartbeat_front_addr":"172.21.15.105:6807/103820","state":["exists","up"]},{"osd":1,"uuid":"9dc43bde-f8d5-45c7-a5a2-5a32399712cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":8,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":103821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6805","nonce":103821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":103821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":103821}]},"public_addr":"172.21.15.105:6804/103821","cluster_addr":"172.21.15.105:6805/103821","heartbeat_back_addr":"172.21.15.105:6808/103821","heartbeat_front_addr":"172.21.15.105:6806/103821","state":["exists","up"]},{"osd":2,"uuid":"e1f033bf-7672-49c7-8caa-57461f6e67b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6803","nonce":103822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":103822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":103822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6811","nonce":103822}]},"public_addr":"172.21.15.105:6803/103822","cluster_addr":"172.21.15.105:6802/103822","heartbeat_back_addr":"172.21.15.105:6810/103822","heartbeat_front_addr":"172.21.15.105:6811/103822","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T14:42:21.407 DEBUG:teuthology.misc:3 of 3 OSDs are up 2023-01-24T14:42:21.407 INFO:tasks.ceph:Creating RBD pool 2023-01-24T14:42:21.407 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T14:42:22.695 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'rbd' created 2023-01-24T14:42:22.708 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T14:42:23.699 INFO:teuthology.orchestra.run.smithi105.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T14:42:23.711 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T14:42:23.712 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T14:42:23.712 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T14:42:23.713 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T14:42:23.713 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T14:42:24.104 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:42:24.117 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":4,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":103413},{"type":"v1","addr":"172.21.15.105:6813","nonce":103413}]},"active_addr":"172.21.15.105:6813/103413","active_change":"2023-01-24T14:42:18.647271+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":3615135081}]},{"addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2929117504}]},{"addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":579344533}]}]}} 2023-01-24T14:42:24.119 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T14:42:24.120 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T14:42:24.120 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T14:42:24.444 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:42:24.445 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":12,"fsid":"6b13f761-45e0-46c8-9934-bb4919ac75a7","created":"2023-01-24T14:42:12.726827+0000","modified":"2023-01-24T14:42:23.691405+0000","last_up_change":"2023-01-24T14:42:18.666231+0000","last_in_change":"2023-01-24T14:42:13.742705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T14:42:18.692732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T14:42:21.752330+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"96902593-f039-4d73-b201-75d8f0bf791a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6801","nonce":103820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":103820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6809","nonce":103820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6807","nonce":103820}]},"public_addr":"172.21.15.105:6801/103820","cluster_addr":"172.21.15.105:6800/103820","heartbeat_back_addr":"172.21.15.105:6809/103820","heartbeat_front_addr":"172.21.15.105:6807/103820","state":["exists","up"]},{"osd":1,"uuid":"9dc43bde-f8d5-45c7-a5a2-5a32399712cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":103821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6805","nonce":103821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":103821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":103821}]},"public_addr":"172.21.15.105:6804/103821","cluster_addr":"172.21.15.105:6805/103821","heartbeat_back_addr":"172.21.15.105:6808/103821","heartbeat_front_addr":"172.21.15.105:6806/103821","state":["exists","up"]},{"osd":2,"uuid":"e1f033bf-7672-49c7-8caa-57461f6e67b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6803","nonce":103822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":103822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":103822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6811","nonce":103822}]},"public_addr":"172.21.15.105:6803/103822","cluster_addr":"172.21.15.105:6802/103822","heartbeat_back_addr":"172.21.15.105:6810/103822","heartbeat_front_addr":"172.21.15.105:6811/103822","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T14:42:24.458 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T14:42:24.459 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T14:42:24.790 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:42:24.790 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":13,"fsid":"6b13f761-45e0-46c8-9934-bb4919ac75a7","created":"2023-01-24T14:42:12.726827+0000","modified":"2023-01-24T14:42:24.694488+0000","last_up_change":"2023-01-24T14:42:18.666231+0000","last_in_change":"2023-01-24T14:42:13.742705+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-01-24T14:42:18.692732+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-01-24T14:42:21.752330+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"96902593-f039-4d73-b201-75d8f0bf791a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6801","nonce":103820}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":103820}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6809","nonce":103820}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6807","nonce":103820}]},"public_addr":"172.21.15.105:6801/103820","cluster_addr":"172.21.15.105:6800/103820","heartbeat_back_addr":"172.21.15.105:6809/103820","heartbeat_front_addr":"172.21.15.105:6807/103820","state":["exists","up"]},{"osd":1,"uuid":"9dc43bde-f8d5-45c7-a5a2-5a32399712cf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":103821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6805","nonce":103821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":103821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":103821}]},"public_addr":"172.21.15.105:6804/103821","cluster_addr":"172.21.15.105:6805/103821","heartbeat_back_addr":"172.21.15.105:6808/103821","heartbeat_front_addr":"172.21.15.105:6806/103821","state":["exists","up"]},{"osd":2,"uuid":"e1f033bf-7672-49c7-8caa-57461f6e67b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6803","nonce":103822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":103822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":103822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6811","nonce":103822}]},"public_addr":"172.21.15.105:6803/103822","cluster_addr":"172.21.15.105:6802/103822","heartbeat_back_addr":"172.21.15.105:6810/103822","heartbeat_front_addr":"172.21.15.105:6811/103822","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T14:42:24.804 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T14:42:24.805 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T14:42:24.806 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T14:42:24.979 INFO:teuthology.orchestra.run.smithi105.stdout:30064771078 2023-01-24T14:42:24.979 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T14:42:24.983 INFO:teuthology.orchestra.run.smithi105.stdout:30064771078 2023-01-24T14:42:24.983 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T14:42:24.984 INFO:teuthology.orchestra.run.smithi105.stdout:30064771078 2023-01-24T14:42:24.985 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T14:42:25.344 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T14:42:25.358 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 0 for osd.1 2023-01-24T14:42:25.371 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T14:42:25.377 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T14:42:25.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 0 for osd.2 2023-01-24T14:42:25.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 0 for osd.0 2023-01-24T14:42:26.359 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T14:42:26.387 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T14:42:26.393 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T14:42:26.712 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T14:42:26.733 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 0 for osd.1 2023-01-24T14:42:26.751 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T14:42:26.755 INFO:teuthology.orchestra.run.smithi105.stdout:0 2023-01-24T14:42:26.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 0 for osd.0 2023-01-24T14:42:26.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 0 for osd.2 2023-01-24T14:42:27.734 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T14:42:27.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T14:42:27.773 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T14:42:28.088 INFO:teuthology.orchestra.run.smithi105.stdout:30064771080 2023-01-24T14:42:28.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 30064771080 for osd.1 2023-01-24T14:42:28.103 DEBUG:teuthology.parallel:result is None 2023-01-24T14:42:28.124 INFO:teuthology.orchestra.run.smithi105.stdout:30064771080 2023-01-24T14:42:28.133 INFO:teuthology.orchestra.run.smithi105.stdout:30064771080 2023-01-24T14:42:28.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 30064771080 for osd.2 2023-01-24T14:42:28.139 DEBUG:teuthology.parallel:result is None 2023-01-24T14:42:28.147 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771078 got 30064771080 for osd.0 2023-01-24T14:42:28.148 DEBUG:teuthology.parallel:result is None 2023-01-24T14:42:28.148 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T14:42:28.149 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T14:42:28.500 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:42:28.503 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-01-24T14:42:28.514 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-01-24T14:42:26.788281+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":29,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":314572800,"kb_used":17208,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":15488,"kb_avail":314555592,"statfs":{"total":322122547200,"available":322104926208,"internally_reserved":0,"allocated":1351680,"data_stored":1236011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15859712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.859398+0000","last_change":"2023-01-24T14:42:23.859398+0000","last_active":"2023-01-24T14:42:23.859398+0000","last_peered":"2023-01-24T14:42:23.859398+0000","last_clean":"2023-01-24T14:42:23.859398+0000","last_became_active":"2023-01-24T14:42:23.847841+0000","last_became_peered":"2023-01-24T14:42:23.847841+0000","last_unstale":"2023-01-24T14:42:23.859398+0000","last_undegraded":"2023-01-24T14:42:23.859398+0000","last_fullsized":"2023-01-24T14:42:23.859398+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.870669+0000","last_change":"2023-01-24T14:42:23.870669+0000","last_active":"2023-01-24T14:42:23.870669+0000","last_peered":"2023-01-24T14:42:23.870669+0000","last_clean":"2023-01-24T14:42:23.870669+0000","last_became_active":"2023-01-24T14:42:23.860735+0000","last_became_peered":"2023-01-24T14:42:23.860735+0000","last_unstale":"2023-01-24T14:42:23.870669+0000","last_undegraded":"2023-01-24T14:42:23.870669+0000","last_fullsized":"2023-01-24T14:42:23.870669+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.877659+0000","last_change":"2023-01-24T14:42:23.877659+0000","last_active":"2023-01-24T14:42:23.877659+0000","last_peered":"2023-01-24T14:42:23.877659+0000","last_clean":"2023-01-24T14:42:23.877659+0000","last_became_active":"2023-01-24T14:42:23.868961+0000","last_became_peered":"2023-01-24T14:42:23.868961+0000","last_unstale":"2023-01-24T14:42:23.877659+0000","last_undegraded":"2023-01-24T14:42:23.877659+0000","last_fullsized":"2023-01-24T14:42:23.877659+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.843909+0000","last_change":"2023-01-24T14:42:23.844817+0000","last_active":"2023-01-24T14:42:23.843909+0000","last_peered":"2023-01-24T14:42:23.843909+0000","last_clean":"2023-01-24T14:42:22.688610+0000","last_became_active":"2023-01-24T14:42:23.841465+0000","last_became_peered":"2023-01-24T14:42:23.841465+0000","last_unstale":"2023-01-24T14:42:23.843909+0000","last_undegraded":"2023-01-24T14:42:23.843909+0000","last_fullsized":"2023-01-24T14:42:23.843909+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":109,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:24.141278+0000","last_change":"2023-01-24T14:42:20.751032+0000","last_active":"2023-01-24T14:42:24.141278+0000","last_peered":"2023-01-24T14:42:24.141278+0000","last_clean":"2023-01-24T14:42:24.141278+0000","last_became_active":"2023-01-24T14:42:20.747912+0000","last_became_peered":"2023-01-24T14:42:20.747912+0000","last_unstale":"2023-01-24T14:42:24.141278+0000","last_undegraded":"2023-01-24T14:42:24.141278+0000","last_fullsized":"2023-01-24T14:42:24.141278+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:19.674827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:19.674827+0000","last_clean_scrub_stamp":"2023-01-24T14:42:19.674827+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":29,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.817218+0000","last_change":"2023-01-24T14:42:23.817218+0000","last_active":"2023-01-24T14:42:23.817218+0000","last_peered":"2023-01-24T14:42:23.817218+0000","last_clean":"2023-01-24T14:42:23.817218+0000","last_became_active":"2023-01-24T14:42:23.812813+0000","last_became_peered":"2023-01-24T14:42:23.812813+0000","last_unstale":"2023-01-24T14:42:23.817218+0000","last_undegraded":"2023-01-24T14:42:23.817218+0000","last_fullsized":"2023-01-24T14:42:23.817218+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.835790+0000","last_change":"2023-01-24T14:42:23.836165+0000","last_active":"2023-01-24T14:42:23.835790+0000","last_peered":"2023-01-24T14:42:23.835790+0000","last_clean":"2023-01-24T14:42:22.688610+0000","last_became_active":"2023-01-24T14:42:23.831749+0000","last_became_peered":"2023-01-24T14:42:23.831749+0000","last_unstale":"2023-01-24T14:42:23.835790+0000","last_undegraded":"2023-01-24T14:42:23.835790+0000","last_fullsized":"2023-01-24T14:42:23.835790+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.808709+0000","last_change":"2023-01-24T14:42:23.809165+0000","last_active":"2023-01-24T14:42:23.808709+0000","last_peered":"2023-01-24T14:42:23.808709+0000","last_clean":"2023-01-24T14:42:22.688610+0000","last_became_active":"2023-01-24T14:42:23.806489+0000","last_became_peered":"2023-01-24T14:42:23.806489+0000","last_unstale":"2023-01-24T14:42:23.808709+0000","last_undegraded":"2023-01-24T14:42:23.808709+0000","last_fullsized":"2023-01-24T14:42:23.808709+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.845409+0000","last_change":"2023-01-24T14:42:23.845409+0000","last_active":"2023-01-24T14:42:23.845409+0000","last_peered":"2023-01-24T14:42:23.845409+0000","last_clean":"2023-01-24T14:42:23.845409+0000","last_became_active":"2023-01-24T14:42:23.822239+0000","last_became_peered":"2023-01-24T14:42:23.822239+0000","last_unstale":"2023-01-24T14:42:23.845409+0000","last_undegraded":"2023-01-24T14:42:23.845409+0000","last_fullsized":"2023-01-24T14:42:23.845409+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":29,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":7,"seq":30064771080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":4924,"kb_used_data":52,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852676,"statfs":{"total":107374182400,"available":107369140224,"internally_reserved":0,"allocated":53248,"data_stored":17662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":7,"seq":30064771080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":6140,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":104851460,"statfs":{"total":107374182400,"available":107367895040,"internally_reserved":0,"allocated":647168,"data_stored":608030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":7,"seq":30064771080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":6144,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":104851456,"statfs":{"total":107374182400,"available":107367890944,"internally_reserved":0,"allocated":651264,"data_stored":610319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-01-24T14:42:28.515 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T14:42:28.845 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:42:28.847 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2023-01-24T14:42:28.858 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-01-24T14:42:28.651929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":29,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":314572800,"kb_used":17208,"kb_used_data":1320,"kb_used_omap":0,"kb_used_meta":15488,"kb_avail":314555592,"statfs":{"total":322122547200,"available":322104926208,"internally_reserved":0,"allocated":1351680,"data_stored":1236011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":15859712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.863648"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.859398+0000","last_change":"2023-01-24T14:42:23.859398+0000","last_active":"2023-01-24T14:42:23.859398+0000","last_peered":"2023-01-24T14:42:23.859398+0000","last_clean":"2023-01-24T14:42:23.859398+0000","last_became_active":"2023-01-24T14:42:23.847841+0000","last_became_peered":"2023-01-24T14:42:23.847841+0000","last_unstale":"2023-01-24T14:42:23.859398+0000","last_undegraded":"2023-01-24T14:42:23.859398+0000","last_fullsized":"2023-01-24T14:42:23.859398+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.870669+0000","last_change":"2023-01-24T14:42:23.870669+0000","last_active":"2023-01-24T14:42:23.870669+0000","last_peered":"2023-01-24T14:42:23.870669+0000","last_clean":"2023-01-24T14:42:23.870669+0000","last_became_active":"2023-01-24T14:42:23.860735+0000","last_became_peered":"2023-01-24T14:42:23.860735+0000","last_unstale":"2023-01-24T14:42:23.870669+0000","last_undegraded":"2023-01-24T14:42:23.870669+0000","last_fullsized":"2023-01-24T14:42:23.870669+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.877659+0000","last_change":"2023-01-24T14:42:23.877659+0000","last_active":"2023-01-24T14:42:23.877659+0000","last_peered":"2023-01-24T14:42:23.877659+0000","last_clean":"2023-01-24T14:42:23.877659+0000","last_became_active":"2023-01-24T14:42:23.868961+0000","last_became_peered":"2023-01-24T14:42:23.868961+0000","last_unstale":"2023-01-24T14:42:23.877659+0000","last_undegraded":"2023-01-24T14:42:23.877659+0000","last_fullsized":"2023-01-24T14:42:23.877659+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.843909+0000","last_change":"2023-01-24T14:42:23.844817+0000","last_active":"2023-01-24T14:42:23.843909+0000","last_peered":"2023-01-24T14:42:23.843909+0000","last_clean":"2023-01-24T14:42:22.688610+0000","last_became_active":"2023-01-24T14:42:23.841465+0000","last_became_peered":"2023-01-24T14:42:23.841465+0000","last_unstale":"2023-01-24T14:42:23.843909+0000","last_undegraded":"2023-01-24T14:42:23.843909+0000","last_fullsized":"2023-01-24T14:42:23.843909+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"12'97","reported_seq":109,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:24.141278+0000","last_change":"2023-01-24T14:42:20.751032+0000","last_active":"2023-01-24T14:42:24.141278+0000","last_peered":"2023-01-24T14:42:24.141278+0000","last_clean":"2023-01-24T14:42:24.141278+0000","last_became_active":"2023-01-24T14:42:20.747912+0000","last_became_peered":"2023-01-24T14:42:20.747912+0000","last_unstale":"2023-01-24T14:42:24.141278+0000","last_undegraded":"2023-01-24T14:42:24.141278+0000","last_fullsized":"2023-01-24T14:42:24.141278+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:19.674827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:19.674827+0000","last_clean_scrub_stamp":"2023-01-24T14:42:19.674827+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":29,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.817218+0000","last_change":"2023-01-24T14:42:23.817218+0000","last_active":"2023-01-24T14:42:23.817218+0000","last_peered":"2023-01-24T14:42:23.817218+0000","last_clean":"2023-01-24T14:42:23.817218+0000","last_became_active":"2023-01-24T14:42:23.812813+0000","last_became_peered":"2023-01-24T14:42:23.812813+0000","last_unstale":"2023-01-24T14:42:23.817218+0000","last_undegraded":"2023-01-24T14:42:23.817218+0000","last_fullsized":"2023-01-24T14:42:23.817218+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.835790+0000","last_change":"2023-01-24T14:42:23.836165+0000","last_active":"2023-01-24T14:42:23.835790+0000","last_peered":"2023-01-24T14:42:23.835790+0000","last_clean":"2023-01-24T14:42:22.688610+0000","last_became_active":"2023-01-24T14:42:23.831749+0000","last_became_peered":"2023-01-24T14:42:23.831749+0000","last_unstale":"2023-01-24T14:42:23.835790+0000","last_undegraded":"2023-01-24T14:42:23.835790+0000","last_fullsized":"2023-01-24T14:42:23.835790+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":8,"reported_epoch":12,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.808709+0000","last_change":"2023-01-24T14:42:23.809165+0000","last_active":"2023-01-24T14:42:23.808709+0000","last_peered":"2023-01-24T14:42:23.808709+0000","last_clean":"2023-01-24T14:42:22.688610+0000","last_became_active":"2023-01-24T14:42:23.806489+0000","last_became_peered":"2023-01-24T14:42:23.806489+0000","last_unstale":"2023-01-24T14:42:23.808709+0000","last_undegraded":"2023-01-24T14:42:23.808709+0000","last_fullsized":"2023-01-24T14:42:23.808709+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":13,"state":"active+clean","last_fresh":"2023-01-24T14:42:23.845409+0000","last_change":"2023-01-24T14:42:23.845409+0000","last_active":"2023-01-24T14:42:23.845409+0000","last_peered":"2023-01-24T14:42:23.845409+0000","last_clean":"2023-01-24T14:42:23.845409+0000","last_became_active":"2023-01-24T14:42:23.822239+0000","last_became_peered":"2023-01-24T14:42:23.822239+0000","last_unstale":"2023-01-24T14:42:23.845409+0000","last_undegraded":"2023-01-24T14:42:23.845409+0000","last_fullsized":"2023-01-24T14:42:23.845409+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T14:42:22.688610+0000","last_clean_scrub_stamp":"2023-01-24T14:42:22.688610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":29,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":2,"up_from":7,"seq":30064771080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":4924,"kb_used_data":52,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852676,"statfs":{"total":107374182400,"available":107369140224,"internally_reserved":0,"allocated":53248,"data_stored":17662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":7,"seq":30064771080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":6140,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":104851460,"statfs":{"total":107374182400,"available":107367895040,"internally_reserved":0,"allocated":647168,"data_stored":608030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":1,"up_from":7,"seq":30064771080,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":6144,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":104851456,"statfs":{"total":107374182400,"available":107367890944,"internally_reserved":0,"allocated":651264,"data_stored":610319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]}],"pool_statfs":[]}} 2023-01-24T14:42:28.859 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T14:42:28.860 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T14:42:28.860 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T14:42:28.860 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T14:42:29.215 INFO:teuthology.orchestra.run.smithi105.stdout: 2023-01-24T14:42:29.216 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T14:42:29.229 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T14:42:29.229 INFO:teuthology.run_tasks:Running task workunit... 2023-01-24T14:42:29.245 INFO:tasks.workunit:Pulling workunits from ref 510284b66513490445619d1430aa869868c71a09 2023-01-24T14:42:29.246 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-01-24T14:42:29.246 DEBUG:teuthology.orchestra.run.smithi105:> stat -- /home/ubuntu/cephtest/mnt.0 2023-01-24T14:42:29.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T14:42:29.264 INFO:teuthology.orchestra.run.smithi105.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-01-24T14:42:29.265 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-01-24T14:42:29.323 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-01-24T14:42:29.323 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-01-24T14:42:29.381 INFO:tasks.workunit:timeout=3h 2023-01-24T14:42:29.381 INFO:tasks.workunit:cleanup=True 2023-01-24T14:42:29.382 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 510284b66513490445619d1430aa869868c71a09 2023-01-24T14:42:29.439 INFO:tasks.workunit.client.0.smithi105.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-01-24T14:43:17.268 INFO:tasks.workunit.client.0.smithi105.stderr:Note: switching to '510284b66513490445619d1430aa869868c71a09'. 2023-01-24T14:43:17.268 INFO:tasks.workunit.client.0.smithi105.stderr: 2023-01-24T14:43:17.268 INFO:tasks.workunit.client.0.smithi105.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-01-24T14:43:17.269 INFO:tasks.workunit.client.0.smithi105.stderr:changes and commit them, and you can discard any commits you make in this 2023-01-24T14:43:17.269 INFO:tasks.workunit.client.0.smithi105.stderr:state without impacting any branches by switching back to a branch. 2023-01-24T14:43:17.269 INFO:tasks.workunit.client.0.smithi105.stderr: 2023-01-24T14:43:17.269 INFO:tasks.workunit.client.0.smithi105.stderr:If you want to create a new branch to retain commits you create, you may 2023-01-24T14:43:17.269 INFO:tasks.workunit.client.0.smithi105.stderr:do so (now or later) by using -c with the switch command. Example: 2023-01-24T14:43:17.270 INFO:tasks.workunit.client.0.smithi105.stderr: 2023-01-24T14:43:17.270 INFO:tasks.workunit.client.0.smithi105.stderr: git switch -c 2023-01-24T14:43:17.270 INFO:tasks.workunit.client.0.smithi105.stderr: 2023-01-24T14:43:17.270 INFO:tasks.workunit.client.0.smithi105.stderr:Or undo this operation with: 2023-01-24T14:43:17.270 INFO:tasks.workunit.client.0.smithi105.stderr: 2023-01-24T14:43:17.271 INFO:tasks.workunit.client.0.smithi105.stderr: git switch - 2023-01-24T14:43:17.271 INFO:tasks.workunit.client.0.smithi105.stderr: 2023-01-24T14:43:17.271 INFO:tasks.workunit.client.0.smithi105.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-01-24T14:43:17.271 INFO:tasks.workunit.client.0.smithi105.stderr: 2023-01-24T14:43:17.271 INFO:tasks.workunit.client.0.smithi105.stderr:HEAD is now at 510284b6651 Merge pull request #49791 from idryomov/wip-doc-exclusive-lock-transitions 2023-01-24T14:43:17.273 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-01-24T14:43:17.346 INFO:tasks.workunit.client.0.smithi105.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-01-24T14:43:17.349 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-24T14:43:17.349 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-01-24T14:43:17.884 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-01-24T14:43:17.952 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-01-24T14:43:17.989 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-24T14:43:17.991 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-24T14:43:17.991 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-01-24T14:43:18.027 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-24T14:43:18.030 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2023-01-24T14:43:18.030 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-01-24T14:43:18.088 INFO:tasks.workunit:Running workunits matching rbd/test_librbd.sh on client.0... 2023-01-24T14:43:18.090 INFO:tasks.workunit:Running workunit rbd/test_librbd.sh... 2023-01-24T14:43:18.090 DEBUG:teuthology.orchestra.run.smithi105:workunit test rbd/test_librbd.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=510284b66513490445619d1430aa869868c71a09 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh 2023-01-24T14:43:18.189 INFO:tasks.workunit.client.0.smithi105.stdout:seed 106457 2023-01-24T14:43:18.190 INFO:tasks.workunit.client.0.smithi105.stdout:[==========] Running 469 tests from 23 test suites. 2023-01-24T14:43:18.190 INFO:tasks.workunit.client.0.smithi105.stdout:[----------] Global test environment set-up. 2023-01-24T14:43:18.190 INFO:tasks.workunit.client.0.smithi105.stdout:[----------] 169 tests from TestLibRBD 2023-01-24T14:43:18.227 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.CreateAndStat 2023-01-24T14:43:19.851 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:22.225 INFO:tasks.workunit.client.0.smithi105.stdout:image has size 2097152 and order 22 2023-01-24T14:43:22.226 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.CreateAndStat (3998 ms) 2023-01-24T14:43:22.226 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.CreateWithSameDataPool 2023-01-24T14:43:22.226 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:43:22.227 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.CreateWithSameDataPool (0 ms) 2023-01-24T14:43:22.227 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.CreateAndStatPP 2023-01-24T14:43:22.227 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:22.523 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.CreateAndStatPP (297 ms) 2023-01-24T14:43:22.523 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.GetId 2023-01-24T14:43:22.524 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:22.814 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.GetId (292 ms) 2023-01-24T14:43:22.815 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.GetIdPP 2023-01-24T14:43:22.815 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:23.077 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.GetIdPP (263 ms) 2023-01-24T14:43:23.078 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.GetBlockNamePrefix 2023-01-24T14:43:23.078 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:23.336 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.GetBlockNamePrefix (259 ms) 2023-01-24T14:43:23.337 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.GetBlockNamePrefixPP 2023-01-24T14:43:23.337 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:23.556 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.GetBlockNamePrefixPP (219 ms) 2023-01-24T14:43:23.556 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestGetCreateTimestamp 2023-01-24T14:43:23.557 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:43:23.557 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestGetCreateTimestamp (0 ms) 2023-01-24T14:43:23.557 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.GetCreateTimestampPP 2023-01-24T14:43:23.557 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:43:23.557 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.GetCreateTimestampPP (0 ms) 2023-01-24T14:43:23.558 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.OpenAio 2023-01-24T14:43:23.558 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:23.776 INFO:tasks.workunit.client.0.smithi105.stdout:image has size 2097152 and order 22 2023-01-24T14:43:23.777 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.OpenAio (221 ms) 2023-01-24T14:43:23.777 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.OpenAioFail 2023-01-24T14:43:23.782 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.OpenAioFail (6 ms) 2023-01-24T14:43:23.783 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.OpenAioPP 2023-01-24T14:43:23.783 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:24.205 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.OpenAioPP (423 ms) 2023-01-24T14:43:24.206 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.OpenAioFailPP 2023-01-24T14:43:24.213 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.OpenAioFailPP (6 ms) 2023-01-24T14:43:24.213 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.ResizeAndStat 2023-01-24T14:43:24.213 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:24.799 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.ResizeAndStat (587 ms) 2023-01-24T14:43:24.799 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.ResizeAndStatPP 2023-01-24T14:43:24.799 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:25.281 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.ResizeAndStatPP (483 ms) 2023-01-24T14:43:25.282 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.UpdateWatchAndResize 2023-01-24T14:43:25.282 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:25.742 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.UpdateWatchAndResize (460 ms) 2023-01-24T14:43:25.742 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.UpdateWatchAndResizePP 2023-01-24T14:43:25.743 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:26.277 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.UpdateWatchAndResizePP (536 ms) 2023-01-24T14:43:26.278 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCreateLsDelete 2023-01-24T14:43:28.442 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:30.141 INFO:tasks.workunit.client.0.smithi105.stdout:image: image15 2023-01-24T14:43:30.142 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image15 2023-01-24T14:43:30.142 INFO:tasks.workunit.client.0.smithi105.stdout:found image15 2023-01-24T14:43:30.142 INFO:tasks.workunit.client.0.smithi105.stdout:erased image15 2023-01-24T14:43:30.142 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:30.819 INFO:tasks.workunit.client.0.smithi105.stdout:image: image15 2023-01-24T14:43:30.819 INFO:tasks.workunit.client.0.smithi105.stdout:image: image16 2023-01-24T14:43:30.819 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image15 2023-01-24T14:43:30.820 INFO:tasks.workunit.client.0.smithi105.stdout:found image15 2023-01-24T14:43:30.820 INFO:tasks.workunit.client.0.smithi105.stdout:erased image15 2023-01-24T14:43:30.820 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image16 2023-01-24T14:43:30.820 INFO:tasks.workunit.client.0.smithi105.stdout:found image16 2023-01-24T14:43:30.820 INFO:tasks.workunit.client.0.smithi105.stdout:erased image16 2023-01-24T14:43:30.821 INFO:tasks.workunit.client.0.smithi105.stdout:image: image16 2023-01-24T14:43:30.821 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image16 2023-01-24T14:43:30.821 INFO:tasks.workunit.client.0.smithi105.stdout:found image16 2023-01-24T14:43:30.821 INFO:tasks.workunit.client.0.smithi105.stdout:erased image16 2023-01-24T14:43:30.821 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCreateLsDelete (4541 ms) 2023-01-24T14:43:30.822 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCreateLsDeletePP 2023-01-24T14:43:31.899 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:34.319 INFO:tasks.workunit.client.0.smithi105.stdout:num images is: 1 2023-01-24T14:43:34.319 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:43:34.320 INFO:tasks.workunit.client.0.smithi105.stdout:image: image17 2023-01-24T14:43:34.320 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image17 2023-01-24T14:43:34.320 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:34.378 INFO:tasks.workunit.client.0.smithi105.stdout:num images is: 2 2023-01-24T14:43:34.379 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 2 2023-01-24T14:43:34.379 INFO:tasks.workunit.client.0.smithi105.stdout:image: image17 2023-01-24T14:43:34.379 INFO:tasks.workunit.client.0.smithi105.stdout:image: image18 2023-01-24T14:43:34.379 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image17 2023-01-24T14:43:34.380 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image18 2023-01-24T14:43:34.660 INFO:tasks.workunit.client.0.smithi105.stdout:num images is: 1 2023-01-24T14:43:34.661 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:43:34.661 INFO:tasks.workunit.client.0.smithi105.stdout:image: image18 2023-01-24T14:43:34.661 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image18 2023-01-24T14:43:34.661 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCreateLsDeletePP (3842 ms) 2023-01-24T14:43:34.662 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCopy 2023-01-24T14:43:35.926 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:43:49.514 INFO:tasks.workunit.client.0.smithi105.stdout:image: image19 2023-01-24T14:43:49.519 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image19 2023-01-24T14:43:49.520 INFO:tasks.workunit.client.0.smithi105.stdout:found image19 2023-01-24T14:43:49.520 INFO:tasks.workunit.client.0.smithi105.stdout:erased image19 2023-01-24T14:43:49.520 INFO:tasks.workunit.client.0.smithi105.stdout:image: image19 2023-01-24T14:43:49.520 INFO:tasks.workunit.client.0.smithi105.stdout:image: image20 2023-01-24T14:43:49.521 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image19 2023-01-24T14:43:49.521 INFO:tasks.workunit.client.0.smithi105.stdout:found image19 2023-01-24T14:43:49.521 INFO:tasks.workunit.client.0.smithi105.stdout:erased image19 2023-01-24T14:43:49.521 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image20 2023-01-24T14:43:49.521 INFO:tasks.workunit.client.0.smithi105.stdout:found image20 2023-01-24T14:43:49.522 INFO:tasks.workunit.client.0.smithi105.stdout:erased image20 2023-01-24T14:43:49.522 INFO:tasks.workunit.client.0.smithi105.stdout:0.00% done 2023-01-24T14:43:49.522 INFO:tasks.workunit.client.0.smithi105.stdout:100.00% done 2023-01-24T14:43:49.522 INFO:tasks.workunit.client.0.smithi105.stdout:image: image19 2023-01-24T14:43:49.522 INFO:tasks.workunit.client.0.smithi105.stdout:image: image20 2023-01-24T14:43:49.523 INFO:tasks.workunit.client.0.smithi105.stdout:image: image21 2023-01-24T14:43:49.523 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image19 2023-01-24T14:43:49.523 INFO:tasks.workunit.client.0.smithi105.stdout:found image19 2023-01-24T14:43:49.523 INFO:tasks.workunit.client.0.smithi105.stdout:erased image19 2023-01-24T14:43:49.523 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image20 2023-01-24T14:43:49.524 INFO:tasks.workunit.client.0.smithi105.stdout:found image20 2023-01-24T14:43:49.524 INFO:tasks.workunit.client.0.smithi105.stdout:erased image20 2023-01-24T14:43:49.524 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image21 2023-01-24T14:43:49.524 INFO:tasks.workunit.client.0.smithi105.stdout:found image21 2023-01-24T14:43:49.524 INFO:tasks.workunit.client.0.smithi105.stdout:erased image21 2023-01-24T14:43:49.525 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCopy (14853 ms) 2023-01-24T14:43:49.525 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCopyPP 2023-01-24T14:43:50.755 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:00.703 INFO:tasks.workunit.client.0.smithi105.stdout:num images is: 1 2023-01-24T14:44:00.704 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:44:00.704 INFO:tasks.workunit.client.0.smithi105.stdout:image: image22 2023-01-24T14:44:00.705 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image22 2023-01-24T14:44:02.567 INFO:tasks.workunit.client.0.smithi105.stdout:num images is: 2 2023-01-24T14:44:02.567 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 2 2023-01-24T14:44:02.567 INFO:tasks.workunit.client.0.smithi105.stdout:image: image22 2023-01-24T14:44:02.567 INFO:tasks.workunit.client.0.smithi105.stdout:image: image23 2023-01-24T14:44:02.568 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image22 2023-01-24T14:44:02.568 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image23 2023-01-24T14:44:03.117 INFO:tasks.workunit.client.0.smithi105.stdout:0.00% done 2023-01-24T14:44:03.117 INFO:tasks.workunit.client.0.smithi105.stdout:100.00% done 2023-01-24T14:44:03.118 INFO:tasks.workunit.client.0.smithi105.stdout:num images is: 3 2023-01-24T14:44:03.119 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 3 2023-01-24T14:44:03.119 INFO:tasks.workunit.client.0.smithi105.stdout:image: image22 2023-01-24T14:44:03.119 INFO:tasks.workunit.client.0.smithi105.stdout:image: image23 2023-01-24T14:44:03.120 INFO:tasks.workunit.client.0.smithi105.stdout:image: image24 2023-01-24T14:44:03.120 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image22 2023-01-24T14:44:03.121 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image23 2023-01-24T14:44:03.121 INFO:tasks.workunit.client.0.smithi105.stdout:expected = image24 2023-01-24T14:44:03.486 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCopyPP (13972 ms) 2023-01-24T14:44:03.486 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestDeepCopy 2023-01-24T14:44:03.486 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:44:03.487 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestDeepCopy (0 ms) 2023-01-24T14:44:03.487 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestDeepCopyPP 2023-01-24T14:44:03.487 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:44:03.487 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestDeepCopyPP (0 ms) 2023-01-24T14:44:03.487 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCreateLsDeleteSnap 2023-01-24T14:44:03.488 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:07.379 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 1 2023-01-24T14:44:07.380 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:44:07.380 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap1 2023-01-24T14:44:07.380 INFO:tasks.workunit.client.0.smithi105.stdout:found snap1 with size 2097152 2023-01-24T14:44:07.380 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 2 2023-01-24T14:44:07.380 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 2 2023-01-24T14:44:07.381 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap1 2023-01-24T14:44:07.381 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap2 2023-01-24T14:44:07.381 INFO:tasks.workunit.client.0.smithi105.stdout:found snap1 with size 2097152 2023-01-24T14:44:07.381 INFO:tasks.workunit.client.0.smithi105.stdout:found snap2 with size 4194304 2023-01-24T14:44:07.381 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 1 2023-01-24T14:44:07.382 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:44:07.382 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap2 2023-01-24T14:44:07.382 INFO:tasks.workunit.client.0.smithi105.stdout:found snap2 with size 4194304 2023-01-24T14:44:07.382 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 0 2023-01-24T14:44:07.382 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 0 2023-01-24T14:44:07.383 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCreateLsDeleteSnap (3894 ms) 2023-01-24T14:44:07.383 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestGetSnapShotTimeStamp 2023-01-24T14:44:07.383 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:44:07.383 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestGetSnapShotTimeStamp (0 ms) 2023-01-24T14:44:07.383 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCreateLsDeleteSnapPP 2023-01-24T14:44:07.383 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:09.353 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 1 2023-01-24T14:44:09.353 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:44:09.354 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap1 2023-01-24T14:44:09.354 INFO:tasks.workunit.client.0.smithi105.stdout:found snap1 with size 2097152 2023-01-24T14:44:10.240 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 2 2023-01-24T14:44:10.241 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 2 2023-01-24T14:44:10.241 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap1 2023-01-24T14:44:10.241 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap2 2023-01-24T14:44:10.241 INFO:tasks.workunit.client.0.smithi105.stdout:found snap1 with size 2097152 2023-01-24T14:44:10.242 INFO:tasks.workunit.client.0.smithi105.stdout:found snap2 with size 4194304 2023-01-24T14:44:11.211 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 1 2023-01-24T14:44:11.211 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:44:11.212 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap2 2023-01-24T14:44:11.212 INFO:tasks.workunit.client.0.smithi105.stdout:found snap2 with size 4194304 2023-01-24T14:44:12.056 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 0 2023-01-24T14:44:12.056 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 0 2023-01-24T14:44:12.102 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCreateLsDeleteSnapPP (4722 ms) 2023-01-24T14:44:12.102 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestGetNameIdSnapPP 2023-01-24T14:44:12.102 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:18.224 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 0 2023-01-24T14:44:18.224 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 0 2023-01-24T14:44:18.270 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestGetNameIdSnapPP (6169 ms) 2023-01-24T14:44:18.270 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCreateLsRenameSnapPP 2023-01-24T14:44:18.271 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:19.096 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 1 2023-01-24T14:44:19.097 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:44:19.097 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap1 2023-01-24T14:44:19.097 INFO:tasks.workunit.client.0.smithi105.stdout:found snap1 with size 2097152 2023-01-24T14:44:20.109 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 2 2023-01-24T14:44:20.109 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 2 2023-01-24T14:44:20.110 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap1 2023-01-24T14:44:20.110 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap2 2023-01-24T14:44:20.110 INFO:tasks.workunit.client.0.smithi105.stdout:found snap1 with size 2097152 2023-01-24T14:44:20.110 INFO:tasks.workunit.client.0.smithi105.stdout:found snap2 with size 4194304 2023-01-24T14:44:20.202 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 2 2023-01-24T14:44:20.202 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 2 2023-01-24T14:44:20.202 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap1-rename 2023-01-24T14:44:20.203 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap2 2023-01-24T14:44:20.203 INFO:tasks.workunit.client.0.smithi105.stdout:found snap1-rename with size 2097152 2023-01-24T14:44:20.203 INFO:tasks.workunit.client.0.smithi105.stdout:found snap2 with size 4194304 2023-01-24T14:44:21.323 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 1 2023-01-24T14:44:21.323 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 1 2023-01-24T14:44:21.323 INFO:tasks.workunit.client.0.smithi105.stdout:snap: snap2-rename 2023-01-24T14:44:21.323 INFO:tasks.workunit.client.0.smithi105.stdout:found snap2-rename with size 4194304 2023-01-24T14:44:22.062 INFO:tasks.workunit.client.0.smithi105.stdout:num snaps is: 0 2023-01-24T14:44:22.062 INFO:tasks.workunit.client.0.smithi105.stdout:expected: 0 2023-01-24T14:44:22.108 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCreateLsRenameSnapPP (3837 ms) 2023-01-24T14:44:22.108 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.ConcurrentCreatesUnvalidatedPool 2023-01-24T14:44:23.866 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:23.866 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:23.866 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:23.867 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:29.614 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.ConcurrentCreatesUnvalidatedPool (7507 ms) 2023-01-24T14:44:29.615 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.RemoveFullTry 2023-01-24T14:44:33.915 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:36.928 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:58.795 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.RemoveFullTry (29180 ms) 2023-01-24T14:44:58.796 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.RemoveFullTryDataPool 2023-01-24T14:44:58.796 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:44:58.796 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.RemoveFullTryDataPool (0 ms) 2023-01-24T14:44:58.797 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestIO 2023-01-24T14:44:58.797 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:44:59.857 INFO:tasks.ceph.osd.1.smithi105.stderr:Aborting on shard 0. 2023-01-24T14:44:59.858 INFO:tasks.ceph.osd.1.smithi105.stderr:Backtrace: 2023-01-24T14:45:01.186 INFO:tasks.ceph.osd.1.smithi105.stderr: 0# gsignal in /lib64/libc.so.6 2023-01-24T14:45:01.187 INFO:tasks.ceph.osd.1.smithi105.stderr: 1# abort in /lib64/libc.so.6 2023-01-24T14:45:01.187 INFO:tasks.ceph.osd.1.smithi105.stderr: 2# ceph::__ceph_assert_fail(char const*, char const*, int, char const*) in ceph-osd 2023-01-24T14:45:01.187 INFO:tasks.ceph.osd.1.smithi105.stderr: 3# crimson::osd::PG::is_degraded_or_backfilling_object(hobject_t const&) const in ceph-osd 2023-01-24T14:45:01.187 INFO:tasks.ceph.osd.1.smithi105.stderr: 4# crimson::osd::CommonClientRequest::do_recover_missing(boost::intrusive_ptr&, hobject_t const&) in ceph-osd 2023-01-24T14:45:01.187 INFO:tasks.ceph.osd.1.smithi105.stderr: 5# 0x000055EA6CC18DCA in ceph-osd 2023-01-24T14:45:01.188 INFO:tasks.ceph.osd.1.smithi105.stderr: 6# 0x000055EA6CC3C3E9 in ceph-osd 2023-01-24T14:45:01.188 INFO:tasks.ceph.osd.1.smithi105.stderr: 7# 0x000055EA6CC3CD61 in ceph-osd 2023-01-24T14:45:01.188 INFO:tasks.ceph.osd.1.smithi105.stderr: 8# 0x000055EA6CC3D589 in ceph-osd 2023-01-24T14:45:01.188 INFO:tasks.ceph.osd.1.smithi105.stderr: 9# 0x000055EA6CC3DC5C in ceph-osd 2023-01-24T14:45:01.188 INFO:tasks.ceph.osd.1.smithi105.stderr:10# auto seastar::internal::future_invoke > ()>&, seastar::internal::monostate>(seastar::noncopyable_function > ()>&, seastar::internal::monostate&&) in ceph-osd 2023-01-24T14:45:01.189 INFO:tasks.ceph.osd.1.smithi105.stderr:11# seastar::continuation, seastar::noncopyable_function > ()>, seastar::future::then_impl_nrvo > ()>, crimson::interruptible::interruptible_future_detail > >(seastar::noncopyable_function > ()>&&)::{lambda(seastar::internal::promise_base_with_type&&, seastar::noncopyable_function > ()>&, seastar::future_state&&)#1}, void>::run_and_dispose() in ceph-osd 2023-01-24T14:45:01.189 INFO:tasks.ceph.osd.1.smithi105.stderr:12# 0x000055EA78C6AC03 in ceph-osd 2023-01-24T14:45:01.189 INFO:tasks.ceph.osd.1.smithi105.stderr:13# 0x000055EA78CC7D86 in ceph-osd 2023-01-24T14:45:01.189 INFO:tasks.ceph.osd.1.smithi105.stderr:14# 0x000055EA78EC1EA8 in ceph-osd 2023-01-24T14:45:01.189 INFO:tasks.ceph.osd.1.smithi105.stderr:15# 0x000055EA78EC3EBF in ceph-osd 2023-01-24T14:45:01.190 INFO:tasks.ceph.osd.1.smithi105.stderr:16# 0x000055EA788042D0 in ceph-osd 2023-01-24T14:45:01.190 INFO:tasks.ceph.osd.1.smithi105.stderr:17# 0x000055EA78808BC2 in ceph-osd 2023-01-24T14:45:01.190 INFO:tasks.ceph.osd.1.smithi105.stderr:18# main in ceph-osd 2023-01-24T14:45:01.190 INFO:tasks.ceph.osd.1.smithi105.stderr:19# __libc_start_main in /lib64/libc.so.6 2023-01-24T14:45:01.190 INFO:tasks.ceph.osd.1.smithi105.stderr:20# _start in ceph-osd 2023-01-24T14:45:01.191 INFO:tasks.ceph.osd.1.smithi105.stderr:Content of /proc/self/maps: 2023-01-24T14:45:01.191 INFO:tasks.ceph.osd.1.smithi105.stderr:7fff7000-8fff7000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.191 INFO:tasks.ceph.osd.1.smithi105.stderr:8fff7000-2008fff7000 ---p 00000000 00:00 0 2023-01-24T14:45:01.191 INFO:tasks.ceph.osd.1.smithi105.stderr:2008fff7000-10007fff8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.191 INFO:tasks.ceph.osd.1.smithi105.stderr:55ea34ca8000-55ea6ab0f000 r--p 00000000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:45:01.191 INFO:tasks.ceph.osd.1.smithi105.stderr:55ea6ab0f000-55ea7a1a1000 r-xp 35e67000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:45:01.192 INFO:tasks.ceph.osd.1.smithi105.stderr:55ea7a1a1000-55ea7c8d9000 r--p 454f9000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:45:01.192 INFO:tasks.ceph.osd.1.smithi105.stderr:55ea7c8da000-55ea7c987000 r--p 47c31000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:45:01.192 INFO:tasks.ceph.osd.1.smithi105.stderr:55ea7c987000-55eab715f000 rw-p 47cde000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:45:01.192 INFO:tasks.ceph.osd.1.smithi105.stderr:55eab715f000-55eab71ce000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.193 INFO:tasks.ceph.osd.1.smithi105.stderr:600000000000-602000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.193 INFO:tasks.ceph.osd.1.smithi105.stderr:602000000000-6020000c0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.193 INFO:tasks.ceph.osd.1.smithi105.stderr:6020000c0000-602e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.193 INFO:tasks.ceph.osd.1.smithi105.stderr:602e00000000-602e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.193 INFO:tasks.ceph.osd.1.smithi105.stderr:602e00010000-603000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.194 INFO:tasks.ceph.osd.1.smithi105.stderr:603000000000-603000640000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.194 INFO:tasks.ceph.osd.1.smithi105.stderr:603000640000-603e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.194 INFO:tasks.ceph.osd.1.smithi105.stderr:603e00000000-603e00070000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.194 INFO:tasks.ceph.osd.1.smithi105.stderr:603e00070000-604000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.194 INFO:tasks.ceph.osd.1.smithi105.stderr:604000000000-604000390000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.195 INFO:tasks.ceph.osd.1.smithi105.stderr:604000390000-604e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.195 INFO:tasks.ceph.osd.1.smithi105.stderr:604e00000000-604e00030000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.195 INFO:tasks.ceph.osd.1.smithi105.stderr:604e00030000-606000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.195 INFO:tasks.ceph.osd.1.smithi105.stderr:606000000000-6060002c0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.195 INFO:tasks.ceph.osd.1.smithi105.stderr:6060002c0000-606e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.195 INFO:tasks.ceph.osd.1.smithi105.stderr:606e00000000-606e00020000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.196 INFO:tasks.ceph.osd.1.smithi105.stderr:606e00020000-607000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.196 INFO:tasks.ceph.osd.1.smithi105.stderr:607000000000-6070001d0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.196 INFO:tasks.ceph.osd.1.smithi105.stderr:6070001d0000-607e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.196 INFO:tasks.ceph.osd.1.smithi105.stderr:607e00000000-607e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.196 INFO:tasks.ceph.osd.1.smithi105.stderr:607e00010000-608000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.197 INFO:tasks.ceph.osd.1.smithi105.stderr:608000000000-608000b60000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.197 INFO:tasks.ceph.osd.1.smithi105.stderr:608000b60000-608e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.197 INFO:tasks.ceph.osd.1.smithi105.stderr:608e00000000-608e00050000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.197 INFO:tasks.ceph.osd.1.smithi105.stderr:608e00050000-60a000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.197 INFO:tasks.ceph.osd.1.smithi105.stderr:60a000000000-60a000030000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.198 INFO:tasks.ceph.osd.1.smithi105.stderr:60a000030000-60ae00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.198 INFO:tasks.ceph.osd.1.smithi105.stderr:60ae00000000-60ae00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.198 INFO:tasks.ceph.osd.1.smithi105.stderr:60ae00010000-60b000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.198 INFO:tasks.ceph.osd.1.smithi105.stderr:60b000000000-60b000360000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.198 INFO:tasks.ceph.osd.1.smithi105.stderr:60b000360000-60be00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.198 INFO:tasks.ceph.osd.1.smithi105.stderr:60be00000000-60be00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.199 INFO:tasks.ceph.osd.1.smithi105.stderr:60be00010000-60c000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.199 INFO:tasks.ceph.osd.1.smithi105.stderr:60c000000000-60c000250000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.199 INFO:tasks.ceph.osd.1.smithi105.stderr:60c000250000-60ce00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.199 INFO:tasks.ceph.osd.1.smithi105.stderr:60ce00000000-60ce00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.199 INFO:tasks.ceph.osd.1.smithi105.stderr:60ce00010000-60d000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.200 INFO:tasks.ceph.osd.1.smithi105.stderr:60d000000000-60d000060000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.200 INFO:tasks.ceph.osd.1.smithi105.stderr:60d000060000-60de00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.200 INFO:tasks.ceph.osd.1.smithi105.stderr:60de00000000-60de00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.200 INFO:tasks.ceph.osd.1.smithi105.stderr:60de00010000-60e000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.200 INFO:tasks.ceph.osd.1.smithi105.stderr:60e000000000-60e0000b0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.201 INFO:tasks.ceph.osd.1.smithi105.stderr:60e0000b0000-60ee00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.201 INFO:tasks.ceph.osd.1.smithi105.stderr:60ee00000000-60ee00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.201 INFO:tasks.ceph.osd.1.smithi105.stderr:60ee00010000-60f000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.201 INFO:tasks.ceph.osd.1.smithi105.stderr:60f000000000-60f0000c0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.201 INFO:tasks.ceph.osd.1.smithi105.stderr:60f0000c0000-60fe00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.201 INFO:tasks.ceph.osd.1.smithi105.stderr:60fe00000000-60fe00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.202 INFO:tasks.ceph.osd.1.smithi105.stderr:60fe00010000-610000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.202 INFO:tasks.ceph.osd.1.smithi105.stderr:610000000000-6100000b0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.202 INFO:tasks.ceph.osd.1.smithi105.stderr:6100000b0000-610e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.202 INFO:tasks.ceph.osd.1.smithi105.stderr:610e00000000-610e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.202 INFO:tasks.ceph.osd.1.smithi105.stderr:610e00010000-611000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.203 INFO:tasks.ceph.osd.1.smithi105.stderr:611000000000-6110001f0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.203 INFO:tasks.ceph.osd.1.smithi105.stderr:6110001f0000-611e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.203 INFO:tasks.ceph.osd.1.smithi105.stderr:611e00000000-611e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.203 INFO:tasks.ceph.osd.1.smithi105.stderr:611e00010000-612000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.203 INFO:tasks.ceph.osd.1.smithi105.stderr:612000000000-612000100000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.204 INFO:tasks.ceph.osd.1.smithi105.stderr:612000100000-612e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.204 INFO:tasks.ceph.osd.1.smithi105.stderr:612e00000000-612e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.204 INFO:tasks.ceph.osd.1.smithi105.stderr:612e00010000-613000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.204 INFO:tasks.ceph.osd.1.smithi105.stderr:613000000000-6130000b0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.204 INFO:tasks.ceph.osd.1.smithi105.stderr:6130000b0000-613e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.204 INFO:tasks.ceph.osd.1.smithi105.stderr:613e00000000-613e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.205 INFO:tasks.ceph.osd.1.smithi105.stderr:613e00010000-614000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.205 INFO:tasks.ceph.osd.1.smithi105.stderr:614000000000-614000190000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.205 INFO:tasks.ceph.osd.1.smithi105.stderr:614000190000-614e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.205 INFO:tasks.ceph.osd.1.smithi105.stderr:614e00000000-614e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.205 INFO:tasks.ceph.osd.1.smithi105.stderr:614e00010000-615000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.206 INFO:tasks.ceph.osd.1.smithi105.stderr:615000000000-6150001e0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.206 INFO:tasks.ceph.osd.1.smithi105.stderr:6150001e0000-615e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.206 INFO:tasks.ceph.osd.1.smithi105.stderr:615e00000000-615e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.206 INFO:tasks.ceph.osd.1.smithi105.stderr:615e00010000-616000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.206 INFO:tasks.ceph.osd.1.smithi105.stderr:616000000000-616000740000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.207 INFO:tasks.ceph.osd.1.smithi105.stderr:616000740000-616e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.207 INFO:tasks.ceph.osd.1.smithi105.stderr:616e00000000-616e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.207 INFO:tasks.ceph.osd.1.smithi105.stderr:616e00010000-617000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.207 INFO:tasks.ceph.osd.1.smithi105.stderr:617000000000-617000120000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.207 INFO:tasks.ceph.osd.1.smithi105.stderr:617000120000-617e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.208 INFO:tasks.ceph.osd.1.smithi105.stderr:617e00000000-617e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.208 INFO:tasks.ceph.osd.1.smithi105.stderr:617e00010000-618000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.208 INFO:tasks.ceph.osd.1.smithi105.stderr:618000000000-6180000e0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.208 INFO:tasks.ceph.osd.1.smithi105.stderr:6180000e0000-618e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.208 INFO:tasks.ceph.osd.1.smithi105.stderr:618e00000000-618e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.208 INFO:tasks.ceph.osd.1.smithi105.stderr:618e00010000-619000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.209 INFO:tasks.ceph.osd.1.smithi105.stderr:619000000000-6190002a0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.209 INFO:tasks.ceph.osd.1.smithi105.stderr:6190002a0000-619e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.209 INFO:tasks.ceph.osd.1.smithi105.stderr:619e00000000-619e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.209 INFO:tasks.ceph.osd.1.smithi105.stderr:619e00010000-61a000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.209 INFO:tasks.ceph.osd.1.smithi105.stderr:61a000000000-61a000100000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.210 INFO:tasks.ceph.osd.1.smithi105.stderr:61a000100000-61ae00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.210 INFO:tasks.ceph.osd.1.smithi105.stderr:61ae00000000-61ae00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.210 INFO:tasks.ceph.osd.1.smithi105.stderr:61ae00010000-61b000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.210 INFO:tasks.ceph.osd.1.smithi105.stderr:61b000000000-61b000060000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.210 INFO:tasks.ceph.osd.1.smithi105.stderr:61b000060000-61be00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.211 INFO:tasks.ceph.osd.1.smithi105.stderr:61be00000000-61be00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.211 INFO:tasks.ceph.osd.1.smithi105.stderr:61be00010000-61c000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.211 INFO:tasks.ceph.osd.1.smithi105.stderr:61c000000000-61c0000e0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.211 INFO:tasks.ceph.osd.1.smithi105.stderr:61c0000e0000-61ce00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.211 INFO:tasks.ceph.osd.1.smithi105.stderr:61ce00000000-61ce00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.211 INFO:tasks.ceph.osd.1.smithi105.stderr:61ce00010000-61d000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.212 INFO:tasks.ceph.osd.1.smithi105.stderr:61d000000000-61d0001b0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.212 INFO:tasks.ceph.osd.1.smithi105.stderr:61d0001b0000-61de00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.212 INFO:tasks.ceph.osd.1.smithi105.stderr:61de00000000-61de00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.212 INFO:tasks.ceph.osd.1.smithi105.stderr:61de00010000-61e000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.212 INFO:tasks.ceph.osd.1.smithi105.stderr:61e000000000-61e0000b0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.213 INFO:tasks.ceph.osd.1.smithi105.stderr:61e0000b0000-61ee00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.213 INFO:tasks.ceph.osd.1.smithi105.stderr:61ee00000000-61ee00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.213 INFO:tasks.ceph.osd.1.smithi105.stderr:61ee00010000-61f000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.213 INFO:tasks.ceph.osd.1.smithi105.stderr:61f000000000-61f000070000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.213 INFO:tasks.ceph.osd.1.smithi105.stderr:61f000070000-61fe00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.214 INFO:tasks.ceph.osd.1.smithi105.stderr:61fe00000000-61fe00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.214 INFO:tasks.ceph.osd.1.smithi105.stderr:61fe00010000-620000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.214 INFO:tasks.ceph.osd.1.smithi105.stderr:620000000000-620000080000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.214 INFO:tasks.ceph.osd.1.smithi105.stderr:620000080000-620e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.214 INFO:tasks.ceph.osd.1.smithi105.stderr:620e00000000-620e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.214 INFO:tasks.ceph.osd.1.smithi105.stderr:620e00010000-621000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.215 INFO:tasks.ceph.osd.1.smithi105.stderr:621000000000-621001480000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.215 INFO:tasks.ceph.osd.1.smithi105.stderr:621001480000-621e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.215 INFO:tasks.ceph.osd.1.smithi105.stderr:621e00000000-621e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.215 INFO:tasks.ceph.osd.1.smithi105.stderr:621e00010000-622000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.215 INFO:tasks.ceph.osd.1.smithi105.stderr:622000000000-6220002d0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.216 INFO:tasks.ceph.osd.1.smithi105.stderr:6220002d0000-622e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.216 INFO:tasks.ceph.osd.1.smithi105.stderr:622e00000000-622e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.216 INFO:tasks.ceph.osd.1.smithi105.stderr:622e00010000-623000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.216 INFO:tasks.ceph.osd.1.smithi105.stderr:623000000000-623000080000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.216 INFO:tasks.ceph.osd.1.smithi105.stderr:623000080000-623e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.217 INFO:tasks.ceph.osd.1.smithi105.stderr:623e00000000-623e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.217 INFO:tasks.ceph.osd.1.smithi105.stderr:623e00010000-624000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.217 INFO:tasks.ceph.osd.1.smithi105.stderr:624000000000-6240007e0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.217 INFO:tasks.ceph.osd.1.smithi105.stderr:6240007e0000-624e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.217 INFO:tasks.ceph.osd.1.smithi105.stderr:624e00000000-624e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.217 INFO:tasks.ceph.osd.1.smithi105.stderr:624e00010000-625000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.218 INFO:tasks.ceph.osd.1.smithi105.stderr:625000000000-6250019b0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.218 INFO:tasks.ceph.osd.1.smithi105.stderr:6250019b0000-625e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.218 INFO:tasks.ceph.osd.1.smithi105.stderr:625e00000000-625e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.218 INFO:tasks.ceph.osd.1.smithi105.stderr:625e00010000-626000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.218 INFO:tasks.ceph.osd.1.smithi105.stderr:626000000000-626000180000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.219 INFO:tasks.ceph.osd.1.smithi105.stderr:626000180000-626e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.219 INFO:tasks.ceph.osd.1.smithi105.stderr:626e00000000-626e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.219 INFO:tasks.ceph.osd.1.smithi105.stderr:626e00010000-627000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.219 INFO:tasks.ceph.osd.1.smithi105.stderr:627000000000-627000040000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.219 INFO:tasks.ceph.osd.1.smithi105.stderr:627000040000-627e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.220 INFO:tasks.ceph.osd.1.smithi105.stderr:627e00000000-627e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.220 INFO:tasks.ceph.osd.1.smithi105.stderr:627e00010000-628000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.220 INFO:tasks.ceph.osd.1.smithi105.stderr:628000000000-628000020000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.220 INFO:tasks.ceph.osd.1.smithi105.stderr:628000020000-628e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.220 INFO:tasks.ceph.osd.1.smithi105.stderr:628e00000000-628e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.220 INFO:tasks.ceph.osd.1.smithi105.stderr:628e00010000-629000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.221 INFO:tasks.ceph.osd.1.smithi105.stderr:629000000000-6290000d0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.221 INFO:tasks.ceph.osd.1.smithi105.stderr:6290000d0000-629e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.221 INFO:tasks.ceph.osd.1.smithi105.stderr:629e00000000-629e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.221 INFO:tasks.ceph.osd.1.smithi105.stderr:629e00010000-62a000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.221 INFO:tasks.ceph.osd.1.smithi105.stderr:62a000000000-62a000100000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.222 INFO:tasks.ceph.osd.1.smithi105.stderr:62a000100000-62ae00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.222 INFO:tasks.ceph.osd.1.smithi105.stderr:62ae00000000-62ae00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.222 INFO:tasks.ceph.osd.1.smithi105.stderr:62ae00010000-62b000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.222 INFO:tasks.ceph.osd.1.smithi105.stderr:62b000000000-62b000010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.222 INFO:tasks.ceph.osd.1.smithi105.stderr:62b000010000-62be00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.223 INFO:tasks.ceph.osd.1.smithi105.stderr:62be00000000-62be00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.223 INFO:tasks.ceph.osd.1.smithi105.stderr:62be00010000-62c000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.223 INFO:tasks.ceph.osd.1.smithi105.stderr:62c000000000-62c000010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.223 INFO:tasks.ceph.osd.1.smithi105.stderr:62c000010000-62ce00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.223 INFO:tasks.ceph.osd.1.smithi105.stderr:62ce00000000-62ce00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.224 INFO:tasks.ceph.osd.1.smithi105.stderr:62ce00010000-62d000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.224 INFO:tasks.ceph.osd.1.smithi105.stderr:62d000000000-62d000190000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.224 INFO:tasks.ceph.osd.1.smithi105.stderr:62d000190000-62de00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.224 INFO:tasks.ceph.osd.1.smithi105.stderr:62de00000000-62de00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.224 INFO:tasks.ceph.osd.1.smithi105.stderr:62de00010000-62e000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.224 INFO:tasks.ceph.osd.1.smithi105.stderr:62e000000000-62e000060000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.225 INFO:tasks.ceph.osd.1.smithi105.stderr:62e000060000-62ee00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.225 INFO:tasks.ceph.osd.1.smithi105.stderr:62ee00000000-62ee00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.225 INFO:tasks.ceph.osd.1.smithi105.stderr:62ee00010000-62f000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.225 INFO:tasks.ceph.osd.1.smithi105.stderr:62f000000000-62f000010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.225 INFO:tasks.ceph.osd.1.smithi105.stderr:62f000010000-62fe00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.226 INFO:tasks.ceph.osd.1.smithi105.stderr:62fe00000000-62fe00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.226 INFO:tasks.ceph.osd.1.smithi105.stderr:62fe00010000-630000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.226 INFO:tasks.ceph.osd.1.smithi105.stderr:630000000000-630000020000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.226 INFO:tasks.ceph.osd.1.smithi105.stderr:630000020000-630e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.226 INFO:tasks.ceph.osd.1.smithi105.stderr:630e00000000-630e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.227 INFO:tasks.ceph.osd.1.smithi105.stderr:630e00010000-631000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.227 INFO:tasks.ceph.osd.1.smithi105.stderr:631000000000-631000fc0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.227 INFO:tasks.ceph.osd.1.smithi105.stderr:631000fc0000-631e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.227 INFO:tasks.ceph.osd.1.smithi105.stderr:631e00000000-631e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.227 INFO:tasks.ceph.osd.1.smithi105.stderr:631e00010000-632000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.228 INFO:tasks.ceph.osd.1.smithi105.stderr:632000000000-632000020000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.228 INFO:tasks.ceph.osd.1.smithi105.stderr:632000020000-632e00000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.228 INFO:tasks.ceph.osd.1.smithi105.stderr:632e00000000-632e00010000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.228 INFO:tasks.ceph.osd.1.smithi105.stderr:632e00010000-640000000000 ---p 00000000 00:00 0 2023-01-24T14:45:01.228 INFO:tasks.ceph.osd.1.smithi105.stderr:640000000000-640000003000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.229 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1cd7c000-7f3f1d05d000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.229 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1d0a0000-7f3f1d1ef000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.229 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1d232000-7f3f1d381000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.229 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1dc24000-7f3f1dc67000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.229 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1dcaa000-7f3f1e1a3000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.229 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1e1e6000-7f3f1e378000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.230 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1e3bb000-7f3f1e7a8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.230 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1e7eb000-7f3f1ece4000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.230 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1ed6a000-7f3f1edad000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.230 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1edf0000-7f3f1efc5000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.230 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1f008000-7f3f1f1dd000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.231 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1f220000-7f3f1f8ab000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.231 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1f8ee000-7f3f1f931000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.231 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f1f974000-7f3f2057e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.231 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f205c1000-7f3f20796000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.231 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f20928000-7f3f20938000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.232 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f20941000-7f3f20b16000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.232 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f20b9c000-7f3f20c22000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.232 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f20c65000-7f3f20ca8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.232 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f20ceb000-7f3f20d2e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.232 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f20db4000-7f3f20df7000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.232 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f20e3a000-7f3f20ec0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.233 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2100f000-7f3f21052000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.233 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f211e4000-7f3f212ad000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.233 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2143f000-7f3f21482000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.233 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f214c5000-7f3f21508000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.233 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2154b000-7f3f2158e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.234 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f21614000-7f3f22717000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.234 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f22751000-7f3f22ee8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.234 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f22ef1000-7f3f23467000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.234 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2346f000-7f3f2363c000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.234 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2365d000-7f3f236c2000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.235 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f236ca000-7f3f23897000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.235 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f238b7000-7f3f23970000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.235 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2397d000-7f3f239ad000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.235 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f239b6000-7f3f23b0c000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.235 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f23b2c000-7f3f23b92000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.236 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f23bb0000-7f3f249e3000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.236 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f249e6000-7f3f24d07000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.236 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f24d28000-7f3f25253000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.236 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2525d000-7f3f25fa2000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.236 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f26030000-7f3f26095000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.237 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2621c000-7f3f262a2000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.237 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2638f000-7f3f26458000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.237 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2649b000-7f3f264de000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.237 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f265a7000-7f3f26739000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.237 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2677c000-7f3f26802000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.237 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f26888000-7f3f2690e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.238 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f269d7000-7f3f26b69000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.238 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f26bac000-7f3f26bef000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.238 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f26c32000-7f3f26f56000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.238 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f26f5f000-7f3f27ce8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.238 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f27d07000-7f3f27d8d000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.239 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f27dd2000-7f3f2802d000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.239 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f28057000-7f3f280dd000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.239 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f280f6000-7f3f28288000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.239 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f282aa000-7f3f282ed000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.239 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f282ed000-7f3f282ee000 ---p 00000000 00:00 0 2023-01-24T14:45:01.240 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f282ee000-7f3f28aee000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.240 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f28af7000-7f3f28b07000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.240 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f28b10000-7f3f28b53000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.240 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f28b8d000-7f3f2969b000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.240 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f296a5000-7f3f2a2f8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.240 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2a2f8000-7f3f2a2f9000 ---p 00000000 00:00 0 2023-01-24T14:45:01.241 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2a2f9000-7f3f2aaf9000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.241 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2aaf9000-7f3f2aafa000 ---p 00000000 00:00 0 2023-01-24T14:45:01.241 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2aafa000-7f3f2b2fa000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.241 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2b2fa000-7f3f2b2fb000 ---p 00000000 00:00 0 2023-01-24T14:45:01.241 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2b2fb000-7f3f2bafb000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.242 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2bb02000-7f3f2bb32000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.242 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2bb35000-7f3f2bbb8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.242 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2bbd9000-7f3f2bc81000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.242 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2bca2000-7f3f2bcd4000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.242 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2bcf3000-7f3f2bde0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.242 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2bdff000-7f3f2bf2f000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.243 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2bf50000-7f3f2c534000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.243 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2c555000-7f3f2c79f000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.243 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2c7a9000-7f3f2c7b9000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.243 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2c7c2000-7f3f2c805000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.243 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2c80f000-7f3f2cbb9000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.244 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2cc3f000-7f3f2cd48000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.244 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2cd49000-7f3f2d0b0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.244 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2d0f3000-7f3f2d179000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.244 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2d285000-7f3f2d30b000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.244 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2d30f000-7f3f2d352000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.245 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2dbae000-7f3f2dbbe000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.245 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2dbc1000-7f3f2dc34000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.245 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2dc7d000-7f3f2dddc000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.245 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2ddde000-7f3f2e6f4000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.245 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2e6f7000-7f3f2e7f0000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.246 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2e7f3000-7f3f2e9f8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.246 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2e9fa000-7f3f2f1e9000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.246 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2f1e9000-7f3f2f1ea000 ---p 00000000 00:00 0 2023-01-24T14:45:01.246 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2f1ea000-7f3f2fa00000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.246 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fa0a000-7f3f2fa3a000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.246 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fa3d000-7f3f2fb04000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.247 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fb07000-7f3f2fb5b000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.247 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fb69000-7f3f2fb89000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.247 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fb99000-7f3f2fba9000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.247 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fbcd000-7f3f2fcf9000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.247 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fd1c000-7f3f2fdc2000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.248 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fde4000-7f3f2fee7000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.248 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2fef1000-7f3f2ff21000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.248 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2ff22000-7f3f2ff42000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.248 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f2ff44000-7f3f2fffa000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.248 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f30002000-7f3f30022000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.249 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f30025000-7f3f300a8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.249 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f300a9000-7f3f3014f000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.249 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f30151000-7f3f30323000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.249 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f30325000-7f3f30355000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.249 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3035f000-7f3f30415000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.249 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f30418000-7f3f30448000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.250 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3044b000-7f3f307bf000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.250 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f307bf000-7f3f307c7000 r--p 00000000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:45:01.250 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f307c7000-7f3f307cc000 r-xp 00008000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:45:01.250 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f307cc000-7f3f307ce000 r--p 0000d000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:45:01.250 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f307ce000-7f3f307cf000 ---p 0000f000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:45:01.250 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f307cf000-7f3f307d0000 r--p 0000f000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:45:01.251 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f307d0000-7f3f307d1000 rw-p 00010000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:45:01.251 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f307d1000-7f3f3083f000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.251 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3083f000-7f3f30840000 ---p 00000000 00:00 0 2023-01-24T14:45:01.251 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f30840000-7f3f31056000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.251 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f31056000-7f3f31057000 ---p 00000000 00:00 0 2023-01-24T14:45:01.252 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f31057000-7f3f3186d000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.252 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3186d000-7f3f3186e000 ---p 00000000 00:00 0 2023-01-24T14:45:01.252 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3186e000-7f3f32084000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.252 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f32084000-7f3f32085000 ---p 00000000 00:00 0 2023-01-24T14:45:01.252 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f32085000-7f3f32885000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.253 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f32890000-7f3f329c4000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.253 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f329c4000-7f3f329d2000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.253 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f329d2000-7f3f329d3000 ---p 00000000 00:00 0 2023-01-24T14:45:01.253 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f329d3000-7f3f331d3000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.253 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f331d3000-7f3f331e4000 rw-s 00000000 00:12 438705 /[aio] (deleted) 2023-01-24T14:45:01.253 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f331e4000-7f3f331fa000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.254 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f331fa000-7f3f331fb000 ---p 00000000 00:00 0 2023-01-24T14:45:01.254 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f331fb000-7f3f339fb000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.254 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f339fb000-7f3f33a0c000 rw-s 00000000 00:12 435060 /[aio] (deleted) 2023-01-24T14:45:01.254 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f33a0c000-7f3f33a22000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.254 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f33a22000-7f3f33a23000 ---p 00000000 00:00 0 2023-01-24T14:45:01.254 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f33a23000-7f3f34239000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.255 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f34239000-7f3f3423a000 ---p 00000000 00:00 0 2023-01-24T14:45:01.255 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3423a000-7f3f34a50000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.255 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f34a50000-7f3f34a51000 ---p 00000000 00:00 0 2023-01-24T14:45:01.255 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f34a51000-7f3f35267000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.255 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f35267000-7f3f35268000 ---p 00000000 00:00 0 2023-01-24T14:45:01.256 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f35268000-7f3f35a7e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.256 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f35a7e000-7f3f35a7f000 ---p 00000000 00:00 0 2023-01-24T14:45:01.256 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f35a7f000-7f3f36295000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.256 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f36295000-7f3f36296000 ---p 00000000 00:00 0 2023-01-24T14:45:01.256 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f36296000-7f3f36aac000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.257 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f36aac000-7f3f36aad000 ---p 00000000 00:00 0 2023-01-24T14:45:01.257 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f36aad000-7f3f37d36000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.257 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f37d36000-7f3f37d37000 ---p 00000000 00:00 0 2023-01-24T14:45:01.257 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f37d37000-7f3f3858a000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.257 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3858a000-7f3f38599000 r--p 00000000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:45:01.257 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38599000-7f3f385c6000 r-xp 0000f000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:45:01.258 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385c6000-7f3f385d5000 r--p 0003c000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:45:01.258 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385d5000-7f3f385d6000 ---p 0004b000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:45:01.258 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385d6000-7f3f385d8000 r--p 0004b000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:45:01.258 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385d8000-7f3f385d9000 rw-p 0004d000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:45:01.258 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385d9000-7f3f385e0000 r--p 00000000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:45:01.259 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385e0000-7f3f385ed000 r-xp 00007000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:45:01.259 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385ed000-7f3f385f1000 r--p 00014000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:45:01.259 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385f1000-7f3f385f2000 ---p 00018000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:45:01.259 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385f2000-7f3f385f3000 r--p 00018000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:45:01.259 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385f3000-7f3f385f4000 rw-p 00019000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:45:01.259 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385f4000-7f3f385ff000 r--p 00000000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:45:01.260 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f385ff000-7f3f3860c000 r-xp 0000b000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:45:01.260 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3860c000-7f3f38611000 r--p 00018000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:45:01.260 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38611000-7f3f38613000 r--p 0001c000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:45:01.260 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38613000-7f3f38614000 rw-p 0001e000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:45:01.260 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38614000-7f3f38623000 r--p 00000000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:45:01.261 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38623000-7f3f3864a000 r-xp 0000f000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:45:01.261 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3864a000-7f3f38657000 r--p 00036000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:45:01.261 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38657000-7f3f38658000 ---p 00043000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:45:01.261 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38658000-7f3f3865a000 r--p 00043000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:45:01.261 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3865a000-7f3f3865b000 rw-p 00045000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:45:01.261 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3865b000-7f3f38662000 r--p 00000000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:45:01.262 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38662000-7f3f3866a000 r-xp 00007000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:45:01.262 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3866a000-7f3f3866d000 r--p 0000f000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:45:01.262 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3866d000-7f3f3866e000 ---p 00012000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:45:01.262 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3866e000-7f3f3866f000 r--p 00012000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:45:01.262 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3866f000-7f3f38670000 rw-p 00013000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:45:01.263 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38670000-7f3f38677000 r--p 00000000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:45:01.263 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38677000-7f3f38682000 r-xp 00007000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:45:01.263 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38682000-7f3f38688000 r--p 00012000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:45:01.263 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38688000-7f3f38689000 ---p 00018000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:45:01.263 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38689000-7f3f3868a000 r--p 00018000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:45:01.263 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3868a000-7f3f3868b000 rw-p 00019000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:45:01.264 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3868b000-7f3f3869a000 r--p 00000000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:45:01.264 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3869a000-7f3f386c6000 r-xp 0000f000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:45:01.264 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f386c6000-7f3f386d5000 r--p 0003b000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:45:01.264 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f386d5000-7f3f386d6000 ---p 0004a000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:45:01.264 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f386d6000-7f3f386d8000 r--p 0004a000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:45:01.265 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f386d8000-7f3f386d9000 rw-p 0004c000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:45:01.265 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f386d9000-7f3f386e1000 r--p 00000000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:45:01.265 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f386e1000-7f3f3870a000 r-xp 00008000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:45:01.265 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3870a000-7f3f38717000 r--p 00031000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:45:01.265 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38717000-7f3f38718000 ---p 0003e000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:45:01.265 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38718000-7f3f38719000 r--p 0003e000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:45:01.266 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38719000-7f3f3871a000 rw-p 0003f000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:45:01.266 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3871a000-7f3f38727000 r--p 00000000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:45:01.266 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38727000-7f3f3877f000 r-xp 0000d000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:45:01.266 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3877f000-7f3f38796000 r--p 00065000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:45:01.266 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38796000-7f3f38798000 r--p 0007b000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:45:01.266 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38798000-7f3f38799000 rw-p 0007d000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:45:01.267 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38799000-7f3f3884b000 r--p 00000000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:45:01.267 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3884b000-7f3f388c5000 r-xp 000b2000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:45:01.267 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f388c5000-7f3f38905000 r--p 0012c000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:45:01.267 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38905000-7f3f38906000 ---p 0016c000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:45:01.267 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38906000-7f3f3890c000 r--p 0016c000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:45:01.268 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3890c000-7f3f3890d000 rw-p 00172000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:45:01.268 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3890d000-7f3f38913000 r--p 00000000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:45:01.268 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38913000-7f3f3891a000 r-xp 00006000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:45:01.268 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3891a000-7f3f3891d000 r--p 0000d000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:45:01.268 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3891d000-7f3f3891e000 r--p 0000f000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:45:01.269 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3891e000-7f3f3891f000 rw-p 00010000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:45:01.269 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3891f000-7f3f38927000 r--p 00000000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:45:01.269 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38927000-7f3f38937000 r-xp 00008000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:45:01.269 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38937000-7f3f3893c000 r--p 00018000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:45:01.269 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3893c000-7f3f3893d000 ---p 0001d000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:45:01.269 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3893d000-7f3f3893e000 r--p 0001d000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:45:01.270 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3893e000-7f3f3893f000 rw-p 0001e000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:45:01.270 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3893f000-7f3f38945000 r--p 00000000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:45:01.270 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38945000-7f3f38950000 r-xp 00006000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:45:01.270 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38950000-7f3f38954000 r--p 00011000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:45:01.270 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38954000-7f3f38955000 r--p 00014000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:45:01.271 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38955000-7f3f38956000 rw-p 00015000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:45:01.271 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38956000-7f3f3895d000 r--p 00000000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:45:01.271 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3895d000-7f3f38963000 r-xp 00007000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:45:01.271 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38963000-7f3f38966000 r--p 0000d000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:45:01.271 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38966000-7f3f38967000 r--p 0000f000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:45:01.271 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38967000-7f3f38968000 rw-p 00010000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:45:01.272 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38968000-7f3f38a1b000 r--p 00000000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:45:01.272 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38a1b000-7f3f38ae9000 r-xp 000b3000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:45:01.272 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38ae9000-7f3f38b3f000 r--p 00181000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:45:01.272 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b3f000-7f3f38b46000 r--p 001d6000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:45:01.272 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b46000-7f3f38b47000 rw-p 001dd000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:45:01.273 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b47000-7f3f38b4d000 r--p 00000000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:45:01.273 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b4d000-7f3f38b53000 r-xp 00006000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:45:01.273 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b53000-7f3f38b56000 r--p 0000c000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:45:01.273 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b56000-7f3f38b57000 r--p 0000e000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:45:01.273 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b57000-7f3f38b58000 rw-p 0000f000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:45:01.273 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b58000-7f3f38b6b000 r-xp 00000000 08:01 98590 /usr/lib64/liboath.so.0.1.3 2023-01-24T14:45:01.274 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38b6b000-7f3f38d6b000 ---p 00013000 08:01 98590 /usr/lib64/liboath.so.0.1.3 2023-01-24T14:45:01.274 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d6b000-7f3f38d6c000 r--p 00013000 08:01 98590 /usr/lib64/liboath.so.0.1.3 2023-01-24T14:45:01.274 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d6c000-7f3f38d74000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.274 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d74000-7f3f38d7c000 r--p 00000000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:45:01.274 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d7c000-7f3f38d88000 r-xp 00008000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:45:01.274 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d88000-7f3f38d8c000 r--p 00014000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:45:01.275 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d8c000-7f3f38d8d000 ---p 00018000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:45:01.275 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d8d000-7f3f38d8e000 r--p 00018000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:45:01.275 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d8e000-7f3f38d8f000 rw-p 00019000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:45:01.275 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d8f000-7f3f38d95000 r--p 00000000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:45:01.275 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d95000-7f3f38d9a000 r-xp 00006000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:45:01.276 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d9a000-7f3f38d9c000 r--p 0000b000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:45:01.276 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d9c000-7f3f38d9d000 ---p 0000d000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:45:01.276 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d9d000-7f3f38d9e000 r--p 0000d000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:45:01.276 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d9e000-7f3f38d9f000 rw-p 0000e000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:45:01.276 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38d9f000-7f3f38da6000 r--p 00000000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:45:01.276 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38da6000-7f3f38dac000 r-xp 00007000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:45:01.277 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38dac000-7f3f38dae000 r--p 0000d000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:45:01.277 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38dae000-7f3f38daf000 ---p 0000f000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:45:01.277 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38daf000-7f3f38db0000 r--p 0000f000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:45:01.277 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38db0000-7f3f38db1000 rw-p 00010000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:45:01.277 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38db1000-7f3f38dc3000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.278 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38dc3000-7f3f38dc4000 r--p 00000000 00:00 0 2023-01-24T14:45:01.278 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38dc4000-7f3f38e2a000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.278 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38e2a000-7f3f38e2b000 ---p 00000000 00:00 0 2023-01-24T14:45:01.278 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f38e2b000-7f3f3962b000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.278 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3962b000-7f3f3963c000 rw-s 00000000 00:12 437467 /[aio] (deleted) 2023-01-24T14:45:01.278 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3963c000-7f3f39675000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.279 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f39675000-7f3f39676000 ---p 00000000 00:00 0 2023-01-24T14:45:01.279 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f39676000-7f3f3deba000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.279 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3deba000-7f3f3debe000 r-xp 00000000 08:01 13845 /usr/lib64/liburcu-common.so.6.0.0 2023-01-24T14:45:01.279 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3debe000-7f3f3e0bd000 ---p 00004000 08:01 13845 /usr/lib64/liburcu-common.so.6.0.0 2023-01-24T14:45:01.279 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e0bd000-7f3f3e0be000 r--p 00003000 08:01 13845 /usr/lib64/liburcu-common.so.6.0.0 2023-01-24T14:45:01.280 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e0be000-7f3f3e0bf000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.280 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e0bf000-7f3f3e0c8000 r-xp 00000000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:45:01.280 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e0c8000-7f3f3e2c7000 ---p 00009000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:45:01.280 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e2c7000-7f3f3e2c8000 r--p 00008000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:45:01.280 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e2c8000-7f3f3e2c9000 rw-p 00009000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:45:01.280 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e2c9000-7f3f3e2d0000 r-xp 00000000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:45:01.281 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e2d0000-7f3f3e4cf000 ---p 00007000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:45:01.281 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e4cf000-7f3f3e4d0000 r--p 00006000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:45:01.281 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e4d0000-7f3f3e4d1000 rw-p 00007000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:45:01.281 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e4d1000-7f3f3e4db000 r-xp 00000000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:45:01.281 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e4db000-7f3f3e6da000 ---p 0000a000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:45:01.282 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e6da000-7f3f3e6db000 r--p 00009000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:45:01.282 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e6db000-7f3f3e6dc000 rw-p 0000a000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:45:01.282 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e6dc000-7f3f3e6ed000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.282 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e6ed000-7f3f3e6f1000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.282 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e6f1000-7f3f3e6f3000 rw-s 00000000 00:0e 14549 anon_inode:[perf_event] 2023-01-24T14:45:01.282 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e6f3000-7f3f3e866000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.283 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e86d000-7f3f3e8ad000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.283 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e8b2000-7f3f3e968000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.283 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e96c000-7f3f3e97c000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.283 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f3e97f000-7f3f4830c000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.283 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4830c000-7f3f48aec000 ---p 00000000 00:00 0 2023-01-24T14:45:01.284 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f48aec000-7f3f48e3e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.284 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f48e3e000-7f3f48ec1000 r-xp 00000000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:45:01.284 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f48ec1000-7f3f490c0000 ---p 00083000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:45:01.284 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f490c0000-7f3f490c1000 r--p 00082000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:45:01.284 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f490c1000-7f3f490c2000 rw-p 00083000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:45:01.284 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f490c2000-7f3f490ca000 r-xp 00000000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:45:01.285 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f490ca000-7f3f492c9000 ---p 00008000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:45:01.285 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f492c9000-7f3f492ca000 r--p 00007000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:45:01.285 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f492ca000-7f3f492cb000 rw-p 00008000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:45:01.285 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f492cb000-7f3f492f2000 r-xp 00000000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:45:01.285 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f492f2000-7f3f494f2000 ---p 00027000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:45:01.286 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f494f2000-7f3f494f3000 r--p 00027000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:45:01.286 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f494f3000-7f3f494f4000 rw-p 00028000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:45:01.286 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f494f4000-7f3f494f6000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.286 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f494f6000-7f3f4958c000 r-xp 00000000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:45:01.286 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4958c000-7f3f4978b000 ---p 00096000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:45:01.286 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4978b000-7f3f4978d000 r--p 00095000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:45:01.287 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4978d000-7f3f4978e000 rw-p 00097000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:45:01.287 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4978e000-7f3f497bd000 r-xp 00000000 08:01 7093 /usr/lib64/libhogweed.so.4.5 2023-01-24T14:45:01.287 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f497bd000-7f3f499bc000 ---p 0002f000 08:01 7093 /usr/lib64/libhogweed.so.4.5 2023-01-24T14:45:01.287 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f499bc000-7f3f499bd000 r--p 0002e000 08:01 7093 /usr/lib64/libhogweed.so.4.5 2023-01-24T14:45:01.287 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f499bd000-7f3f499be000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.287 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f499be000-7f3f499f5000 r-xp 00000000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:45:01.288 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f499f5000-7f3f49bf5000 ---p 00037000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:45:01.288 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49bf5000-7f3f49bf7000 r--p 00037000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:45:01.288 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49bf7000-7f3f49bf8000 rw-p 00039000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:45:01.288 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49bf8000-7f3f49c0a000 r-xp 00000000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:45:01.288 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49c0a000-7f3f49e09000 ---p 00012000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:45:01.289 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49e09000-7f3f49e0a000 r--p 00011000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:45:01.289 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49e0a000-7f3f49e0b000 rw-p 00012000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:45:01.289 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49e0b000-7f3f49f88000 r-xp 00000000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:45:01.289 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f49f88000-7f3f4a187000 ---p 0017d000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:45:01.289 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a187000-7f3f4a18b000 r--p 0017c000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:45:01.289 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a18b000-7f3f4a18c000 rw-p 00180000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:45:01.290 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a18c000-7f3f4a1a8000 r-xp 00000000 08:01 6924 /usr/lib64/libidn2.so.0.3.6 2023-01-24T14:45:01.290 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a1a8000-7f3f4a3a8000 ---p 0001c000 08:01 6924 /usr/lib64/libidn2.so.0.3.6 2023-01-24T14:45:01.290 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a3a8000-7f3f4a3a9000 r--p 0001c000 08:01 6924 /usr/lib64/libidn2.so.0.3.6 2023-01-24T14:45:01.290 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a3a9000-7f3f4a3aa000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.290 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a3aa000-7f3f4a4c1000 r-xp 00000000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:45:01.290 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a4c1000-7f3f4a6c0000 ---p 00117000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:45:01.291 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a6c0000-7f3f4a6ca000 r--p 00116000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:45:01.291 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a6ca000-7f3f4a6d4000 rw-p 00120000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:45:01.291 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a6d4000-7f3f4a72a000 r-xp 00000000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:45:01.291 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a72a000-7f3f4a929000 ---p 00056000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:45:01.291 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a929000-7f3f4a92c000 r--p 00055000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:45:01.292 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a92c000-7f3f4a92d000 rw-p 00058000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:45:01.292 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a92d000-7f3f4a92e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.292 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a92e000-7f3f4a934000 r-xp 00000000 08:01 533 /usr/lib64/libuuid.so.1.3.0 2023-01-24T14:45:01.292 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4a934000-7f3f4ab34000 ---p 00006000 08:01 533 /usr/lib64/libuuid.so.1.3.0 2023-01-24T14:45:01.292 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ab34000-7f3f4ab35000 r--p 00006000 08:01 533 /usr/lib64/libuuid.so.1.3.0 2023-01-24T14:45:01.293 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ab35000-7f3f4ab36000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.293 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ab36000-7f3f4acf2000 r-xp 00000000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:45:01.293 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4acf2000-7f3f4aef2000 ---p 001bc000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:45:01.293 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4aef2000-7f3f4aef6000 r--p 001bc000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:45:01.293 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4aef6000-7f3f4aef8000 rw-p 001c0000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:45:01.293 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4aef8000-7f3f4aefc000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.294 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4aefc000-7f3f4af13000 r-xp 00000000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:45:01.294 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4af13000-7f3f4b112000 ---p 00017000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:45:01.294 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4b112000-7f3f4b113000 r--p 00016000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:45:01.294 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4b113000-7f3f4b114000 rw-p 00017000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:45:01.294 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4b114000-7f3f4b160000 r-xp 00000000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:45:01.295 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4b160000-7f3f4b35f000 ---p 0004c000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:45:01.295 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4b35f000-7f3f4b361000 r--p 0004b000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:45:01.295 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4b361000-7f3f4b364000 rw-p 0004d000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:45:01.295 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4b364000-7f3f4be36000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.295 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4be36000-7f3f4bfb7000 r-xp 00000000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:45:01.296 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4bfb7000-7f3f4c1b6000 ---p 00181000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:45:01.296 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c1b6000-7f3f4c1b7000 r--p 00180000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:45:01.296 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c1b7000-7f3f4c1b8000 rw-p 00181000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:45:01.296 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c1b8000-7f3f4c33e000 r-xp 00000000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:45:01.296 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c33e000-7f3f4c53d000 ---p 00186000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:45:01.296 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c53d000-7f3f4c549000 r--p 00185000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:45:01.297 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c549000-7f3f4c54a000 rw-p 00191000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:45:01.297 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c54a000-7f3f4c54d000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.297 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c54d000-7f3f4c568000 r-xp 00000000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:45:01.297 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c568000-7f3f4c767000 ---p 0001b000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:45:01.297 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c767000-7f3f4c768000 r--p 0001a000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:45:01.297 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c768000-7f3f4c769000 rw-p 0001b000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:45:01.298 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c769000-7f3f4c76d000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.298 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c76d000-7f3f4c778000 r-xp 00000000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:45:01.298 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c778000-7f3f4c977000 ---p 0000b000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:45:01.298 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c977000-7f3f4c978000 r--p 0000a000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:45:01.298 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c978000-7f3f4c979000 rw-p 0000b000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:45:01.298 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c979000-7f3f4c9ed000 r-xp 00000000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:45:01.299 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4c9ed000-7f3f4cbed000 ---p 00074000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:45:01.299 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cbed000-7f3f4cbef000 r--p 00074000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:45:01.299 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cbef000-7f3f4cbf0000 rw-p 00076000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:45:01.299 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cbf0000-7f3f4cbf1000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.299 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cbf1000-7f3f4cbf8000 r-xp 00000000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:45:01.300 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cbf8000-7f3f4cdf7000 ---p 00007000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:45:01.300 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cdf7000-7f3f4cdf8000 r--p 00006000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:45:01.300 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cdf8000-7f3f4cdf9000 rw-p 00007000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:45:01.300 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cdf9000-7f3f4cdfc000 r-xp 00000000 08:01 13879 /usr/lib64/libsctp.so.1.0.18 2023-01-24T14:45:01.300 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cdfc000-7f3f4cffb000 ---p 00003000 08:01 13879 /usr/lib64/libsctp.so.1.0.18 2023-01-24T14:45:01.300 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cffb000-7f3f4cffc000 r--p 00002000 08:01 13879 /usr/lib64/libsctp.so.1.0.18 2023-01-24T14:45:01.301 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cffc000-7f3f4cffd000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.301 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4cffd000-7f3f4d003000 r-xp 00000000 08:01 98588 /usr/lib64/libatomic.so.1.2.0 2023-01-24T14:45:01.301 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d003000-7f3f4d202000 ---p 00006000 08:01 98588 /usr/lib64/libatomic.so.1.2.0 2023-01-24T14:45:01.301 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d202000-7f3f4d203000 r--p 00005000 08:01 98588 /usr/lib64/libatomic.so.1.2.0 2023-01-24T14:45:01.301 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d203000-7f3f4d205000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.302 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d205000-7f3f4d3e2000 r-xp 00000000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:45:01.302 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d3e2000-7f3f4d5e1000 ---p 001dd000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:45:01.302 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d5e1000-7f3f4d5f2000 r--p 001dc000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:45:01.302 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d5f2000-7f3f4d5f4000 rw-p 001ed000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:45:01.302 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d5f4000-7f3f4d5f6000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.303 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d5f6000-7f3f4d5f9000 r-xp 00000000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:45:01.303 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d5f9000-7f3f4d7f8000 ---p 00003000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:45:01.303 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d7f8000-7f3f4d7f9000 r--p 00002000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:45:01.303 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d7f9000-7f3f4d7fa000 rw-p 00003000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:45:01.303 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4d7fa000-7f3f4dcb7000 r-xp 00000000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:45:01.303 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4dcb7000-7f3f4deb6000 ---p 004bd000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:45:01.304 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4deb6000-7f3f4df02000 r--p 004bc000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:45:01.304 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4df02000-7f3f4df03000 rw-p 00508000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:45:01.304 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4df03000-7f3f4df07000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.304 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4df07000-7f3f4df19000 r-xp 00000000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:45:01.304 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4df19000-7f3f4e118000 ---p 00012000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:45:01.304 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e118000-7f3f4e119000 r--p 00011000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:45:01.305 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e119000-7f3f4e11a000 rw-p 00012000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:45:01.305 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e11a000-7f3f4e3ce000 r-xp 00000000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:45:01.305 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e3ce000-7f3f4e5ce000 ---p 002b4000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:45:01.305 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e5ce000-7f3f4e5fa000 r--p 002b4000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:45:01.305 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e5fa000-7f3f4e5fe000 rw-p 002e0000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:45:01.306 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e5fe000-7f3f4e603000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.306 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e603000-7f3f4e60a000 r-xp 00000000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:45:01.306 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e60a000-7f3f4e809000 ---p 00007000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:45:01.306 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e809000-7f3f4e80a000 r--p 00006000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:45:01.306 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e80a000-7f3f4e80b000 rw-p 00007000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:45:01.306 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e80b000-7f3f4e80d000 r-xp 00000000 08:01 7079 /usr/lib64/libaio.so.1.0.1 2023-01-24T14:45:01.307 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4e80d000-7f3f4ea0c000 ---p 00002000 08:01 7079 /usr/lib64/libaio.so.1.0.1 2023-01-24T14:45:01.307 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ea0c000-7f3f4ea0d000 r--p 00001000 08:01 7079 /usr/lib64/libaio.so.1.0.1 2023-01-24T14:45:01.307 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ea0d000-7f3f4ea0e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.307 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ea0e000-7f3f4eaa0000 r-xp 00000000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:45:01.307 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eaa0000-7f3f4eca0000 ---p 00092000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:45:01.307 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eca0000-7f3f4eca4000 r--p 00092000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:45:01.308 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eca4000-7f3f4eca5000 rw-p 00096000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:45:01.308 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eca5000-7f3f4ecf2000 r-xp 00000000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:45:01.308 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ecf2000-7f3f4eef1000 ---p 0004d000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:45:01.308 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eef1000-7f3f4eef6000 r--p 0004c000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:45:01.308 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eef6000-7f3f4eef7000 rw-p 00051000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:45:01.308 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eef7000-7f3f4eef8000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.309 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4eef8000-7f3f4ef0e000 r-xp 00000000 08:01 6611 /usr/lib64/libz.so.1.2.11 2023-01-24T14:45:01.309 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ef0e000-7f3f4f10e000 ---p 00016000 08:01 6611 /usr/lib64/libz.so.1.2.11 2023-01-24T14:45:01.309 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f10e000-7f3f4f10f000 r--p 00016000 08:01 6611 /usr/lib64/libz.so.1.2.11 2023-01-24T14:45:01.309 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f10f000-7f3f4f110000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.309 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f110000-7f3f4f12c000 r-xp 00000000 08:01 7082 /usr/lib64/liblz4.so.1.8.3 2023-01-24T14:45:01.310 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f12c000-7f3f4f32b000 ---p 0001c000 08:01 7082 /usr/lib64/liblz4.so.1.8.3 2023-01-24T14:45:01.310 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f32b000-7f3f4f32c000 r--p 0001b000 08:01 7082 /usr/lib64/liblz4.so.1.8.3 2023-01-24T14:45:01.310 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f32c000-7f3f4f32d000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.310 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f32d000-7f3f4f336000 r-xp 00000000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:45:01.310 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f336000-7f3f4f536000 ---p 00009000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:45:01.310 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f536000-7f3f4f537000 r--p 00009000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:45:01.311 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f537000-7f3f4f538000 rw-p 0000a000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:45:01.311 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f538000-7f3f4f55c000 r--p 00000000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:45:01.311 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f55c000-7f3f4f633000 r-xp 00024000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:45:01.311 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f633000-7f3f4f665000 r--p 000fb000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:45:01.311 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f665000-7f3f4f666000 ---p 0012d000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:45:01.312 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f666000-7f3f4f66a000 r--p 0012d000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:45:01.312 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f66a000-7f3f4f66d000 rw-p 00131000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:45:01.312 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4f66d000-7f3f4ff21000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.312 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ff21000-7f3f4ff4e000 r-xp 00000000 08:01 6471 /usr/lib64/ld-2.28.so 2023-01-24T14:45:01.312 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f4ff4e000-7f3f50146000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.313 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f50146000-7f3f5014e000 rw-p 00000000 00:00 0 2023-01-24T14:45:01.313 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f5014e000-7f3f5014f000 r--p 0002d000 08:01 6471 /usr/lib64/ld-2.28.so 2023-01-24T14:45:01.313 INFO:tasks.ceph.osd.1.smithi105.stderr:7f3f5014f000-7f3f50151000 rw-p 0002e000 08:01 6471 /usr/lib64/ld-2.28.so 2023-01-24T14:45:01.313 INFO:tasks.ceph.osd.1.smithi105.stderr:7ffc11f28000-7ffc11fd9000 rw-p 00000000 00:00 0 [stack] 2023-01-24T14:45:01.313 INFO:tasks.ceph.osd.1.smithi105.stderr:7ffc11fec000-7ffc11ff0000 r--p 00000000 00:00 0 [vvar] 2023-01-24T14:45:01.313 INFO:tasks.ceph.osd.1.smithi105.stderr:7ffc11ff0000-7ffc11ff2000 r-xp 00000000 00:00 0 [vdso] 2023-01-24T14:45:01.314 INFO:tasks.ceph.osd.1.smithi105.stderr:ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] 2023-01-24T14:45:01.442 INFO:tasks.ceph.osd.1.smithi105.stderr:daemon-helper: command crashed with signal 6 2023-01-24T14:45:03.711 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-01-24T14:45:09.114 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~5s 2023-01-24T14:45:14.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~11s 2023-01-24T14:45:19.920 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~16s 2023-01-24T14:45:25.323 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~22s 2023-01-24T14:45:30.726 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~27s 2023-01-24T14:45:36.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~32s 2023-01-24T14:45:38.458 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:38.458 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:38.459 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:38.459 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:38.459 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:38.459 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.459 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:38.460 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.460 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.460 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:38.460 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.460 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:38.461 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.461 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.461 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:38.461 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.461 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:38.462 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.462 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.462 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:38.462 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.462 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:38.463 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.463 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.463 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:38.463 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.463 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:38.463 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.464 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.464 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:38.464 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:38.464 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:38.464 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:38.465 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:38.465 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:38.465 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:38.465 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:38.465 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:38.465 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:38.466 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:38.466 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.466 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:38.466 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.466 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.466 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:38.467 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.467 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:38.467 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.467 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.467 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:38.468 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.468 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:38.468 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.468 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.468 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:38.469 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.469 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:38.469 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.469 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.469 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:38.469 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.470 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:38.470 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.470 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.470 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:38.470 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.470 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.471 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.471 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.471 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.471 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.471 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:38.472 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:38.472 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:38.472 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.472 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:38.472 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:38.472 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:38.473 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.473 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:38.473 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:38.473 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:38.473 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.474 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:38.474 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:38.474 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:38.474 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.474 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:38.474 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:38.475 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:38.475 INFO:tasks.workunit.client.0.smithi105.stdout:discard: 512~512 = 512 2023-01-24T14:45:38.475 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.475 INFO:tasks.workunit.client.0.smithi105.stdout:aio discard: 1536~512 = 0 2023-01-24T14:45:38.475 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.475 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.476 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.476 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.476 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:38.476 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:45:38.476 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.476 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.477 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:45:38.477 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.477 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.477 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:45:38.477 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.478 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.478 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:45:38.478 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.478 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.478 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.478 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.479 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:45:38.479 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.479 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.479 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:45:38.479 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.479 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.480 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:45:38.480 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.480 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.480 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:45:38.481 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.481 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.481 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.481 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.481 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:45:38.481 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.482 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.482 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:45:38.482 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.482 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.482 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:45:38.482 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.483 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.483 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:45:38.483 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.483 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.483 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.484 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.484 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:45:38.484 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.484 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.484 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:45:38.485 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.485 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.485 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:45:38.485 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.485 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.485 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:45:38.486 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.486 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.486 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.486 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.486 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:45:38.487 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.487 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.487 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:45:38.487 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.487 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.487 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:45:38.488 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.488 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.488 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:45:38.488 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.488 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.489 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.489 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.489 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.489 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.489 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.489 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.490 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:38.490 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.490 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.490 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.490 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.490 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.491 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.491 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:38.491 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.491 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.491 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.491 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.492 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.492 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.492 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:38.492 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.492 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.493 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.493 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.493 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.493 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.493 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:38.493 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.494 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.494 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.494 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.494 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.494 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.495 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.495 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:38.495 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.495 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.495 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.496 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.496 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:38.496 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.496 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.496 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:38.497 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:38.497 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:38.497 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:38.497 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:38.497 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:38.498 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:38.498 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:41.531 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~38s 2023-01-24T14:45:44.283 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.283 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.283 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.284 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.284 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.284 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.284 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.284 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.285 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.285 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.285 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.285 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.285 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.286 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.286 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.286 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.286 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.286 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.287 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.287 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.287 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.287 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.287 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.288 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.288 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.288 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.288 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.288 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.288 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.289 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.289 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.289 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.289 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.289 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.290 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.290 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.290 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.290 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.290 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.291 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.291 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.291 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.291 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.291 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.291 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.292 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.292 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.292 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.292 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.292 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.293 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.293 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.293 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.293 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.293 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.294 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.294 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.294 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.294 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.294 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.294 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.295 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.295 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.295 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.295 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.295 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.296 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.296 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.296 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.296 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.296 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.297 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.297 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.297 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.297 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.297 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.297 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.298 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.298 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.298 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.298 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.298 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.299 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.299 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.299 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.299 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.299 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.300 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.300 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.300 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.300 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.300 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.300 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.301 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.301 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.301 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.301 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.301 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.301 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.302 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.302 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.302 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.302 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.302 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.303 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.303 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.303 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.303 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.303 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.304 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.304 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:44.304 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:44.304 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:44.304 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:44.305 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:44.305 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.305 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.305 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.305 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:44.305 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:44.306 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:44.306 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:44.306 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:44.306 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:44.306 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:44.306 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestIO (45488 ms) 2023-01-24T14:45:44.307 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestEncryptionLUKS1 2023-01-24T14:45:44.307 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:45:46.934 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~43s 2023-01-24T14:45:52.337 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~49s 2023-01-24T14:45:54.975 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:54.976 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:54.976 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:54.976 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:54.976 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:45:54.976 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.977 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:54.977 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.977 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.977 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:54.977 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.977 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:54.978 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.978 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.978 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:54.978 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.978 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:54.979 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.979 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.979 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:54.979 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.979 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:54.979 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.980 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.980 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:54.980 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.980 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:45:54.980 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.981 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.981 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:45:54.981 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:54.981 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:54.981 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:54.981 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:54.982 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:54.982 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:54.982 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:54.982 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:54.982 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:45:54.982 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:45:54.983 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.983 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:54.983 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.983 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.983 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:54.984 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.984 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:54.984 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.984 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.984 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:54.985 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.985 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:54.985 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.985 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.985 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:54.985 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.986 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:54.986 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.986 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.986 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:54.986 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.986 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:45:54.987 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.987 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:54.987 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:45:54.987 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.987 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.987 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.988 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.988 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.988 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.988 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:54.988 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:54.989 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:54.989 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.989 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:54.989 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:54.989 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:54.989 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.990 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:54.990 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:54.990 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:54.990 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.990 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:54.990 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:54.991 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:54.991 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:54.991 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:45:54.991 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:45:54.991 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:45:54.992 INFO:tasks.workunit.client.0.smithi105.stdout:discard: 512~512 = 512 2023-01-24T14:45:54.992 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:54.992 INFO:tasks.workunit.client.0.smithi105.stdout:aio discard: 1536~512 = 0 2023-01-24T14:45:54.992 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.992 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.992 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.993 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.993 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:45:54.993 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:45:54.993 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.993 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.994 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:45:54.994 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.994 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.994 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:45:54.994 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.995 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.995 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:45:54.995 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.995 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.995 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:54.996 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:54.996 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:45:54.996 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.996 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.996 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:45:54.997 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.997 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.997 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:45:54.997 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.997 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.997 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:45:54.998 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.998 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.998 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:54.998 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:54.998 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:45:54.999 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.999 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:54.999 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:45:54.999 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:54.999 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.000 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:45:55.000 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.000 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.000 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:45:55.000 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.000 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.001 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.001 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.001 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:45:55.001 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.001 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.002 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:45:55.002 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.002 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.002 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:45:55.002 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.002 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.003 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:45:55.003 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.003 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.003 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.003 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.004 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:45:55.004 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.004 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.004 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:45:55.004 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.005 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.005 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:45:55.005 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.005 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.005 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:45:55.005 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.006 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.006 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.006 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.006 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.006 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.007 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:55.007 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:55.007 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:55.007 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.007 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.008 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.008 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.008 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:55.008 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:55.008 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:55.008 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.009 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.009 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.009 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.009 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:55.009 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:55.010 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:55.010 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.010 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.010 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.010 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.011 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:55.011 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:55.011 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:55.011 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.011 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.012 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.012 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.012 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.012 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.012 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.012 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:45:55.013 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.013 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.013 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:55.013 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:55.013 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:55.013 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.014 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.014 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:55.014 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:45:55.014 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:45:55.014 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:45:55.015 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:45:55.015 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:45:55.015 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:45:55.015 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:45:57.740 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~54s 2023-01-24T14:46:03.142 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~59s 2023-01-24T14:46:05.871 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.871 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.872 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.872 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.872 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.872 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.873 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.873 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.873 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.873 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.873 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.873 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.874 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.874 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.874 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.874 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.874 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.875 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.875 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.875 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.875 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.875 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.875 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.876 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.876 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.876 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.876 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.876 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.877 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.877 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.877 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.877 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.877 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.878 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.878 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.878 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.878 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.878 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.879 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.879 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.879 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.879 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.879 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.880 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.880 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.880 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.880 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.880 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.881 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.881 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.881 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.881 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.881 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.882 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.882 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.882 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.882 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.882 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.883 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.883 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.883 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.883 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.883 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.884 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.884 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.884 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.884 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.884 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.885 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.885 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.885 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.885 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.886 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.886 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.886 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.886 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.886 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.887 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.887 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.887 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.887 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.887 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.888 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.888 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.888 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.888 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.888 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.889 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.889 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.889 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.889 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.889 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.889 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.890 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.890 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.890 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.890 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.890 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.891 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.891 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.891 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.891 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.891 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.891 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.892 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.892 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.892 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.892 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.892 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.893 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.893 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:05.893 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:05.893 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:05.893 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:05.894 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:05.894 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.894 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.894 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.894 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:05.894 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:05.895 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:05.895 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:05.895 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:05.895 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:46:05.895 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:05.896 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 4 2023-01-24T14:46:05.896 INFO:tasks.workunit.client.0.smithi105.stdout:read: 4 2023-01-24T14:46:05.896 INFO:tasks.workunit.client.0.smithi105.stdout:read: 4 2023-01-24T14:46:05.896 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestEncryptionLUKS1 (21588 ms) 2023-01-24T14:46:05.896 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestEncryptionLUKS2 2023-01-24T14:46:05.896 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:46:08.545 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~65s 2023-01-24T14:46:13.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~70s 2023-01-24T14:46:19.351 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~76s 2023-01-24T14:46:19.887 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:46:19.887 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:46:19.887 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:46:19.888 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:46:19.888 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:46:19.888 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.888 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:46:19.888 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.888 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.889 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:46:19.889 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.889 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:46:19.889 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.889 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.890 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:46:19.890 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.890 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:46:19.890 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.890 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.890 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:46:19.891 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.891 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:46:19.891 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.891 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.891 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:46:19.892 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.892 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:46:19.892 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.892 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.892 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:46:19.892 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:46:19.893 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:46:19.893 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:46:19.893 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:46:19.893 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:46:19.893 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:46:19.894 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:46:19.894 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:46:19.894 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:46:19.894 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:46:19.894 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.894 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:46:19.895 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.895 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.895 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:46:19.895 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.895 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:46:19.896 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.896 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.896 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:46:19.896 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.896 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:46:19.896 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.897 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.897 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:46:19.897 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.897 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:46:19.897 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.898 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.898 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:46:19.898 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.898 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:46:19.898 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.898 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.899 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:46:19.899 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.899 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.899 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.899 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.899 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.900 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.900 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:46:19.900 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:19.900 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:46:19.900 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.901 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:46:19.901 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:19.901 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:46:19.901 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.901 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:46:19.901 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:19.902 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:46:19.902 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.902 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:46:19.902 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:19.902 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:46:19.902 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.903 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:46:19.903 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:19.903 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:46:19.903 INFO:tasks.workunit.client.0.smithi105.stdout:discard: 512~512 = 512 2023-01-24T14:46:19.903 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.904 INFO:tasks.workunit.client.0.smithi105.stdout:aio discard: 1536~512 = 0 2023-01-24T14:46:19.904 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.904 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.904 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.904 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.904 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:46:19.905 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:46:19.905 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.905 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.905 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:46:19.905 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.906 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.906 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:46:19.906 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.906 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.906 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:46:19.906 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.907 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.907 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.907 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.907 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:46:19.907 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.908 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.908 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:46:19.908 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.908 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.908 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:46:19.908 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.909 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.909 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:46:19.909 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.909 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.909 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.909 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.910 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:46:19.910 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.910 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.910 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:46:19.910 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.910 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.911 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:46:19.911 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.911 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.911 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:46:19.911 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.912 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.912 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.912 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.912 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:46:19.912 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.912 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.913 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:46:19.913 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.913 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.913 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:46:19.913 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.914 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.914 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:46:19.914 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.914 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.914 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.914 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.915 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:46:19.915 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.915 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.915 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:46:19.915 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.915 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.916 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:46:19.916 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.916 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.916 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:46:19.916 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.916 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.917 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.917 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.917 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.917 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.917 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.917 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.918 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:19.918 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.918 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.918 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.918 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.919 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.919 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.919 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:19.919 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.919 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.919 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.920 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.920 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.920 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.920 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:19.920 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.920 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.921 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.921 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.921 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.921 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.921 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:19.922 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.922 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.922 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.922 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.922 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.922 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.923 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.923 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:19.923 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.923 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.923 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.923 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.924 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:19.924 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.924 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.924 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:19.924 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:19.925 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:19.925 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:19.925 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:19.925 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:19.925 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:19.925 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:24.753 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~81s 2023-01-24T14:46:29.842 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.842 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.842 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.843 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.843 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.843 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.843 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.843 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.844 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.844 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.844 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.844 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.844 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.845 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.845 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.845 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.845 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.845 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.846 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.846 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.846 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.846 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.846 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.847 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.847 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.847 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.847 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.847 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.848 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.848 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.848 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.848 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.848 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.849 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.849 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.849 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.849 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.849 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.849 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.850 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.850 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.850 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.850 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.850 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.851 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.851 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.851 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.851 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.851 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.851 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.852 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.852 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.852 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.852 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.852 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.853 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.853 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.853 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.853 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.853 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.854 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.854 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.854 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.854 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.854 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.855 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.855 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.855 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.855 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.855 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.855 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.856 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.856 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.856 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.856 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.856 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.857 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.857 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.857 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.857 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.857 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.858 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.858 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.858 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.858 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.858 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.859 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.859 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.859 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.859 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.859 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.859 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.860 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.860 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.860 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.860 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.860 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.861 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.861 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.861 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.861 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.861 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.861 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.862 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.862 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.862 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.862 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.862 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.863 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.863 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.863 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:46:29.863 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:46:29.863 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:46:29.863 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:46:29.864 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:46:29.864 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.864 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.864 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.864 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:46:29.865 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:46:29.865 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:46:29.865 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:29.865 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:29.865 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:46:29.865 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:46:29.866 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 4 2023-01-24T14:46:29.866 INFO:tasks.workunit.client.0.smithi105.stdout:read: 4 2023-01-24T14:46:29.866 INFO:tasks.workunit.client.0.smithi105.stdout:read: 4 2023-01-24T14:46:29.866 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestEncryptionLUKS2 (23971 ms) 2023-01-24T14:46:29.866 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCloneEncryption 2023-01-24T14:46:29.866 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:46:29.867 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCloneEncryption (0 ms) 2023-01-24T14:46:29.867 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.LUKS1UnderLUKS2WithoutResize 2023-01-24T14:46:29.867 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:46:29.867 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.LUKS1UnderLUKS2WithoutResize (0 ms) 2023-01-24T14:46:29.867 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.LUKS2UnderLUKS1WithoutResize 2023-01-24T14:46:29.868 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:46:29.868 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.LUKS2UnderLUKS1WithoutResize (0 ms) 2023-01-24T14:46:29.868 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.EncryptionFormatNoData 2023-01-24T14:46:29.868 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:46:30.170 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~86s 2023-01-24T14:46:35.573 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~92s 2023-01-24T14:46:40.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~97s 2023-01-24T14:46:46.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~103s 2023-01-24T14:46:47.248 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.EncryptionFormatNoData (17406 ms) 2023-01-24T14:46:47.248 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.EncryptionLoadBadSize 2023-01-24T14:46:47.248 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:46:51.782 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~108s 2023-01-24T14:46:52.820 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.EncryptionLoadBadSize (5572 ms) 2023-01-24T14:46:52.820 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.EncryptionLoadBadStripePattern 2023-01-24T14:46:52.820 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:46:52.821 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.EncryptionLoadBadStripePattern (0 ms) 2023-01-24T14:46:52.821 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.EncryptionLoadFormatMismatch 2023-01-24T14:46:52.821 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:46:52.821 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.EncryptionLoadFormatMismatch (0 ms) 2023-01-24T14:46:52.821 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.EncryptedResize 2023-01-24T14:46:52.822 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:46:56.633 INFO:tasks.ceph.osd.0.smithi105.stderr:Reactor stalled for 38 ms on shard 0. Backtrace: 0x45d4d 0x442cc6e9 0x43fe48bb 0x43ffefad 0x43fff454 0x43fff6e8 0x43fff9b7 0x12cef 0x2acf0 0x2b4c3 0x2b832 0x28adf 0xb4614 0x445f3527 0x445f875d 0x36aba437 0x3618baa7 0x43fc2c02 0x4401fd85 0x44219ea7 0x4421bebe 0x43b5c2cf 0x43b60bc1 0x361d2ca4 0x3ad84 0x35e7229d 2023-01-24T14:46:56.634 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.634 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac51f69e 0xffffffffac521b06 0xffffffffacdf08e9 0xffffffffacdf0ed5 0xffffffffac52604a 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.634 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.634 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.634 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.635 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.635 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.635 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.635 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.635 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.636 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.636 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.636 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.636 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.636 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.637 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6e0a2c 0xffffffffac6e1126 0xffffffffac6e36d3 0xffffffffac6e3edb 0xffffffffac6bbed4 0xffffffffac6e1369 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.637 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.637 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffc02232ff 2023-01-24T14:46:56.637 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.637 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.638 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.638 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.638 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.638 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.638 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.639 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac73f486 0xffffffffac744e1f 0xffffffffac698963 0xffffffffac698a1a 0xffffffffac698f0f 0xffffffffac69ae6f 0xffffffffc05cb62e 0xffffffffac6966d2 0xffffffffac69b9c2 0xffffffffc05b78e5 0xffffffffac75d462 0xffffffffac760c55 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.639 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.639 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacdf6e91 0xffffffffac6a7541 0xffffffffac6e4e84 0xffffffffac6e50ec 0xffffffffac6dfc51 0xffffffffac6e1e6d 0xffffffffac6e2158 0xffffffffac6e21e7 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.639 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.639 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac762560 0xffffffffac760cf9 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.640 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.640 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.640 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.640 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.640 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.641 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac743d12 0xffffffffac744e2f 0xffffffffac6d96e2 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.641 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.641 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.641 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.641 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6d5110 0xffffffffac6d9683 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.642 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffc05c31e7 0xffffffffc05cb5cf 0xffffffffac6966d2 0xffffffffac69b9c2 0xffffffffc05b78e5 0xffffffffac75d462 0xffffffffac760c55 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.642 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.642 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.642 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.642 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.643 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.643 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.643 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.643 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.643 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.644 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.644 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.644 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.644 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.644 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.645 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.645 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaccd8580 0xffffffffaccd8ce0 0xffffffffaccdc73a 0xffffffffaccdf25b 0xffffffffaccebc37 0xffffffffaccee1fb 0xffffffffaccc074c 0xffffffffaccc093d 0xffffffffaccc09bb 0xffffffffaccc0cba 0xffffffffacc24e15 0xffffffffacc26a1a 0xffffffffacc263dd 0xffffffffacc268a2 0xffffffffad0000dc 0xfffffffface00f7a 0xffffffffac4fa465 0xffffffffac4fa4bf 0xffffffffaccc49c6 0xffffffffaccc6490 0xffffffffaccc5ecd 0xffffffffacce35b6 0xffffffffacce4a9d 0xffffffffacce5952 0xffffffffaccd1000 0xffffffffaccd1167 0xffffffffacbfa642 0xffffffffacbfa9ba 0xffffffffacbfcd1c 0xffffffffacbfcde7 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.645 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.645 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.645 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.646 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6bc68c 0xffffffffac741760 0xffffffffac7417de 0xffffffffac6ebfcb 0xffffffffac6d97c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.646 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.646 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.646 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.646 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.647 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.647 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.647 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.647 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.647 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.648 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.648 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.648 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacddf837 0xffffffffac6f6712 0xffffffffac6f9378 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.648 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.648 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6cd811 0xffffffffac6e07f8 0xffffffffac6e1bc3 0xffffffffac6e1d42 0xffffffffac6e2158 0xffffffffac6e21e7 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.649 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.649 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.649 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.649 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.649 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.649 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.650 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacdf6e91 0xffffffffac546f19 0xffffffffac7b9325 0xffffffffac76196e 0xffffffffac5177ea 0xffffffffac403d1f 0xffffffffac4044ec 0xfffffffface000a9 2023-01-24T14:46:56.650 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c9ab 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.650 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.650 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.650 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.651 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.651 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.651 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6ec1c4 0xffffffffac6d75c9 0xffffffffac6d8140 0xffffffffac6dfc28 0xffffffffac6e1e6d 0xffffffffac6e2158 0xffffffffac6e21e7 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.651 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac726c19 0xffffffffc03846aa 0xffffffffc05ab236 0xffffffffc05ccf81 0xffffffffac69671e 0xffffffffac69b9c2 0xffffffffc05b78e5 0xffffffffac75d462 0xffffffffac760c55 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.651 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.651 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.652 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.652 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.652 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.652 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.652 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6d753a 0xffffffffac6d8140 0xffffffffac6dfc28 0xffffffffac6e1e6d 0xffffffffac6e2158 0xffffffffac6e21e7 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.653 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac73e4cc 0xffffffffac744e61 0xffffffffac6d96e2 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.653 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.653 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.653 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6dbee6 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.653 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.654 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.654 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.654 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.654 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.654 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.655 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.655 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac5a701d 0xffffffffac5a7705 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:46:56.655 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.655 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacddf837 0xffffffffac6f6712 0xffffffffac6f9378 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.655 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c919 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.656 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.656 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.656 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.656 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacddf837 0xffffffffac6f6712 0xffffffffac6f9378 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.656 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.657 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.657 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c919 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.657 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.657 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.657 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.657 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.658 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.658 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacddf837 0xffffffffac6f6712 0xffffffffac6f9378 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:46:56.658 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.658 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:56.658 INFO:tasks.ceph.osd.0.smithi105.stderr:kernel callstack: 2023-01-24T14:46:57.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~113s 2023-01-24T14:47:02.588 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~119s 2023-01-24T14:47:06.775 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.EncryptedResize (13955 ms) 2023-01-24T14:47:06.776 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.EncryptedFlattenSmallData 2023-01-24T14:47:06.776 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:47:06.776 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.EncryptedFlattenSmallData (0 ms) 2023-01-24T14:47:06.776 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestIOWithIOHint 2023-01-24T14:47:06.776 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:47:07.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~124s 2023-01-24T14:47:13.394 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~130s 2023-01-24T14:47:14.002 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:47:14.002 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:47:14.002 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:47:14.002 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:47:14.003 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:47:14.003 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.003 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:47:14.003 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.003 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.003 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:47:14.004 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.004 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:47:14.004 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.004 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.004 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:47:14.005 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.005 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:47:14.005 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.005 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.005 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:47:14.006 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.006 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:47:14.006 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.006 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.006 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:47:14.006 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.007 INFO:tasks.workunit.client.0.smithi105.stdout:started write 2023-01-24T14:47:14.007 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.007 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.007 INFO:tasks.workunit.client.0.smithi105.stdout:finished write 2023-01-24T14:47:14.007 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:47:14.008 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:47:14.008 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:47:14.008 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:47:14.008 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:47:14.008 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:47:14.009 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:47:14.009 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:47:14.009 INFO:tasks.workunit.client.0.smithi105.stdout:start compare and write 2023-01-24T14:47:14.009 INFO:tasks.workunit.client.0.smithi105.stdout:compare and wrote: 512 2023-01-24T14:47:14.009 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.009 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:47:14.010 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.010 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.010 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:47:14.010 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.010 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:47:14.011 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.011 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.011 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:47:14.011 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.011 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:47:14.011 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.012 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.012 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:47:14.012 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.012 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:47:14.012 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.012 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.013 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:47:14.013 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.013 INFO:tasks.workunit.client.0.smithi105.stdout:started aio compare and write 2023-01-24T14:47:14.013 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.013 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.014 INFO:tasks.workunit.client.0.smithi105.stdout:finished aio compare and write 2023-01-24T14:47:14.014 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.014 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.014 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.014 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.014 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.015 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.015 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:47:14.015 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:47:14.015 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:47:14.015 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.015 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:47:14.016 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:47:14.016 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:47:14.016 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.016 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:47:14.016 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:47:14.017 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:47:14.017 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.017 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:47:14.017 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:47:14.017 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:47:14.017 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.018 INFO:tasks.workunit.client.0.smithi105.stdout:started read 2023-01-24T14:47:14.018 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:47:14.018 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 512 2023-01-24T14:47:14.018 INFO:tasks.workunit.client.0.smithi105.stdout:discard: 512~512 = 512 2023-01-24T14:47:14.018 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.019 INFO:tasks.workunit.client.0.smithi105.stdout:aio discard: 1536~512 = 0 2023-01-24T14:47:14.019 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.019 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.019 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.019 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.019 INFO:tasks.workunit.client.0.smithi105.stdout:read: 512 2023-01-24T14:47:14.020 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:47:14.020 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.020 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.020 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 0 2023-01-24T14:47:14.020 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.021 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.021 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:47:14.021 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.021 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.021 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 16384 2023-01-24T14:47:14.021 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.022 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.022 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.022 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.022 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:47:14.022 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.022 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.023 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 49152 2023-01-24T14:47:14.023 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.023 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.023 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:47:14.023 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.024 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.024 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 65536 2023-01-24T14:47:14.024 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.024 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.024 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.024 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.025 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:47:14.025 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.025 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.025 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 98304 2023-01-24T14:47:14.025 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.026 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.026 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:47:14.026 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.026 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.026 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 114688 2023-01-24T14:47:14.026 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.027 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.027 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.027 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.027 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:47:14.027 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.028 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.028 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 147456 2023-01-24T14:47:14.028 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.028 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.028 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:47:14.028 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.029 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.029 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 163840 2023-01-24T14:47:14.029 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.029 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.029 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.029 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.030 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:47:14.030 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.030 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.030 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 196608 2023-01-24T14:47:14.030 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.031 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.031 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:47:14.031 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.031 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.031 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 212992 2023-01-24T14:47:14.031 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.032 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.032 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.032 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.032 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.032 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.033 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.033 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.033 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:14.033 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.033 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.033 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.034 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.034 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.034 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.034 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:14.034 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.035 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.035 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.035 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.035 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.035 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.035 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:14.036 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.036 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.036 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.036 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.036 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.036 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.037 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:14.037 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.037 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.037 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.037 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.037 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.038 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.038 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.038 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:14.038 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.038 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.039 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.039 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.039 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:14.039 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.039 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.039 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:14.040 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:14.040 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:14.040 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:14.040 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:14.040 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:14.041 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:14.041 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:18.797 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~135s 2023-01-24T14:47:19.780 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.780 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.780 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.781 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.781 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.781 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.781 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.781 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.782 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.782 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.782 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.782 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.782 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.783 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.783 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.783 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.783 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.783 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.783 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.784 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.784 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.784 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.784 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.784 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.785 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.785 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.785 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.785 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.785 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.786 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.786 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.786 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.786 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.786 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.786 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.787 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.787 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.787 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.787 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.787 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.788 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.788 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.788 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.788 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.788 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.789 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.789 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.789 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.789 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.789 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.790 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.790 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.790 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.790 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.790 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.790 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.791 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.791 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.791 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.791 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.791 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.792 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.792 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.792 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.792 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.792 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.793 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.793 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.793 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.793 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.793 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.794 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.794 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.794 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.794 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.794 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.794 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.795 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.795 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.795 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.795 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.795 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.795 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.796 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.796 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.796 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.796 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.796 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.797 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.797 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.797 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.797 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.797 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.798 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.798 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.798 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.798 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.798 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.799 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.799 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.799 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.799 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.799 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.800 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.800 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.800 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.800 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.800 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.801 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.801 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.801 INFO:tasks.workunit.client.0.smithi105.stdout:write completion cb called! 2023-01-24T14:47:19.801 INFO:tasks.workunit.client.0.smithi105.stdout:return value is: 0 2023-01-24T14:47:19.801 INFO:tasks.workunit.client.0.smithi105.stdout:finished writesame 2023-01-24T14:47:19.801 INFO:tasks.workunit.client.0.smithi105.stdout:to verify the data 2023-01-24T14:47:19.802 INFO:tasks.workunit.client.0.smithi105.stdout:verified 2023-01-24T14:47:19.802 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.802 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.802 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.802 INFO:tasks.workunit.client.0.smithi105.stdout:created completion 2023-01-24T14:47:19.802 INFO:tasks.workunit.client.0.smithi105.stdout:started writesame 2023-01-24T14:47:19.803 INFO:tasks.workunit.client.0.smithi105.stdout:expected fail, finished writesame 2023-01-24T14:47:19.803 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:47:19.803 INFO:tasks.workunit.client.0.smithi105.stdout:wrote: 512 2023-01-24T14:47:19.803 INFO:tasks.workunit.client.0.smithi105.stdout:read completion cb called! 2023-01-24T14:47:19.803 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestIOWithIOHint (13005 ms) 2023-01-24T14:47:19.804 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestDataPoolIO 2023-01-24T14:47:19.804 INFO:tasks.workunit.client.0.smithi105.stdout:SKIPPING 2023-01-24T14:47:19.804 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestDataPoolIO (0 ms) 2023-01-24T14:47:19.804 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteMismatch 2023-01-24T14:47:19.804 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:47:19.864 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCompareAndWriteMismatch (84 ms) 2023-01-24T14:47:19.864 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteMismatch 2023-01-24T14:47:19.865 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:47:20.126 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteMismatch (263 ms) 2023-01-24T14:47:20.127 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteSuccess 2023-01-24T14:47:20.127 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:47:20.254 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCompareAndWriteSuccess (127 ms) 2023-01-24T14:47:20.254 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteSuccess 2023-01-24T14:47:20.254 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:47:20.305 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestAioCompareAndWriteSuccess (51 ms) 2023-01-24T14:47:20.305 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestCompareAndWriteStripeUnitUnaligned 2023-01-24T14:47:20.305 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:47:20.420 INFO:tasks.workunit.client.0.smithi105.stdout:[ OK ] TestLibRBD.TestCompareAndWriteStripeUnitUnaligned (115 ms) 2023-01-24T14:47:20.420 INFO:tasks.workunit.client.0.smithi105.stdout:[ RUN ] TestLibRBD.TestAioCompareAndWriteStripeUnitUnaligned 2023-01-24T14:47:20.421 INFO:tasks.workunit.client.0.smithi105.stdout:using old format 2023-01-24T14:47:20.538 INFO:tasks.ceph.osd.2.smithi105.stderr:Aborting on shard 0. 2023-01-24T14:47:20.539 INFO:tasks.ceph.osd.2.smithi105.stderr:Backtrace: 2023-01-24T14:47:20.611 INFO:tasks.ceph.osd.2.smithi105.stderr:Reactor stalled for 72 ms on shard 0. Backtrace: 0x45d4d 0x442cc6e9 0x43fe48bb 0x43ffefad 0x43fff454 0x43fff6e8 0x43fff9b7 0x12cef 0xfaa62 0x3c0fc762 0x3c1001f6 0x3c103d6d 0x3c1049de 0x3c1050ac 0x3c0f8e37 0x3c0f931b 0x3c0f98c0 0x12cef 0x4eafe 0x21ea4 0x3bfefe00 0x36ae1216 0x38115705 0x37f70dc9 0x37f943e8 0x37f94d60 0x37f95588 0x37f95c5b 0x380acbcd 0x380ad0c0 0x43fc2c02 0x4401fd85 0x44219ea7 0x4421bebe 0x43b5c2cf 0x43b60bc1 0x361d2ca4 0x3ad84 0x35e7229d 2023-01-24T14:47:20.611 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac40436e 0xfffffffface000a9 2023-01-24T14:47:20.611 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c919 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.612 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.612 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.612 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac7a3cf3 0xffffffffc05ccf65 0xffffffffac69671e 0xffffffffac69b9c2 0xffffffffc05b78e5 0xffffffffac75d462 0xffffffffac760c55 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.612 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.612 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.613 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c919 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.613 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacbfcbd0 0xffffffffacbfcc73 0xffffffffacbfcd04 0xffffffffacbfcde7 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.613 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.613 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6d3608 0xffffffffac5a39e6 0xffffffffac5a4368 0xffffffffac5a6e16 0xffffffffac5a7705 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.613 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffc0383dfb 0xffffffffc05ab12e 0xffffffffc05cb704 0xffffffffac6966d2 0xffffffffac69b9c2 0xffffffffc05b78e5 0xffffffffac75d462 0xffffffffac760c55 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.613 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac743d12 0xffffffffac744e2f 0xffffffffac6d96e2 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.614 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.614 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.614 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac7a3491 0xffffffffac7a36c4 0xffffffffc05c4f44 0xffffffffc05c8a22 0xffffffffc05c8cb1 0xffffffffc05ce099 0xffffffffac795291 0xffffffffac7a3d67 0xffffffffc05ccf65 0xffffffffac69671e 0xffffffffac69b9c2 0xffffffffc05b78e5 0xffffffffac75d462 0xffffffffac760c55 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.614 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.614 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.615 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.615 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.615 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.615 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.615 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.615 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.616 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.616 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.616 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac734d00 0xffffffffac6e0744 0xffffffffac6e1278 0xffffffffac6e641b 0xffffffffac6e6734 0xffffffffac6e68fb 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.616 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.616 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacddf837 0xffffffffac6f6712 0xffffffffac6f9378 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.617 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.617 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.617 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.617 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.617 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.618 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac7259fd 0xffffffffac7b646e 0xffffffffac7b6e4e 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.618 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.618 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.618 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.618 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.619 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.619 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.619 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffc0330e5c 0xffffffffacc263dd 0xffffffffacc268a2 0xffffffffad0000dc 0xffffffffac4faaa5 0xffffffffac4faaba 0xfffffffface01e9f 0xfffffffface00a8f 2023-01-24T14:47:20.619 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.619 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.619 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.620 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.620 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.620 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.620 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.620 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.621 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.621 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.621 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.621 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.621 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.622 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.622 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.622 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.622 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.622 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6ccf7c 0xffffffffac6dfe06 0xffffffffac47c83a 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.622 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.623 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.623 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.623 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.623 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.623 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.624 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.624 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.624 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.624 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacddf837 0xffffffffac6f6712 0xffffffffac6f9378 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.624 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffc05c426e 0xffffffffc05c4435 0xffffffffc05c4ad2 0xffffffffc05c8972 0xffffffffc05c8cd7 0xffffffffc05ce099 0xffffffffac795291 0xffffffffac7a3d67 0xffffffffc05ccf65 0xffffffffac69671e 0xffffffffac69b9c2 0xffffffffc05b78e5 0xffffffffac75d462 0xffffffffac760c55 0xffffffffac760edf 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.625 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.625 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac743d12 0xffffffffac744e2f 0xffffffffac6d96e2 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.625 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.625 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6fbabe 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.625 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.625 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.626 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.626 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.626 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.626 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.626 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.627 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.627 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.627 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.627 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c919 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.627 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.628 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac40436e 0xfffffffface000a9 2023-01-24T14:47:20.628 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.628 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.628 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.628 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.628 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.629 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.629 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.629 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.629 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.629 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c919 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.630 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.630 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.630 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.630 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.630 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.631 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6f862c 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.631 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.631 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.631 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47fbc2 0xffffffffac6e6280 0xffffffffac6e646b 0xffffffffac6e6734 0xffffffffac6e68fb 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.631 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.631 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.632 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.632 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.632 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.632 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.632 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.633 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.633 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.633 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47fca9 0xffffffffac6e4ef9 0xffffffffac6e50ec 0xffffffffac6d827a 0xffffffffac6fd957 0xffffffffac6ff106 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:20.633 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.633 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.634 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.634 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.634 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.634 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.634 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.634 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.635 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.635 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac47c919 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.635 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.635 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.635 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6dc5d4 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.636 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffacddf839 0xffffffffac6f6712 0xffffffffac6f9378 0xffffffffac6fbac6 0xffffffffac7173d4 0xffffffffac6d96c4 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.636 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6a8c29 0xffffffffac6d97cf 0xffffffffac6dc2f3 0xffffffffac6dc62a 0xffffffffac47c8b0 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.636 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.636 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:20.636 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac556ba1 0xffffffffac47c8c4 0xffffffffac47cb47 0xfffffffface0114e 2023-01-24T14:47:20.775 INFO:tasks.ceph.osd.2.smithi105.stderr:Reactor stalled for 237 ms on shard 0. Backtrace: 0x45d4d 0x442cc6e9 0x43fe48bb 0x43ffefad 0x43fff454 0x43fff6e8 0x43fff9b7 0x12cef 0x169729 0x3c1011fa 0x3c103d6d 0x3c1049de 0x3c1050ac 0x3c0f8e37 0x3c0f931b 0x3c0f98c0 0x12cef 0x4eafe 0x21ea4 0x3bfefe00 0x36ae1216 0x38115705 0x37f70dc9 0x37f943e8 0x37f94d60 0x37f95588 0x37f95c5b 0x380acbcd 0x380ad0c0 0x43fc2c02 0x4401fd85 0x44219ea7 0x4421bebe 0x43b5c2cf 0x43b60bc1 0x361d2ca4 0x3ad84 0x35e7229d 2023-01-24T14:47:20.775 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 2023-01-24T14:47:21.011 INFO:tasks.ceph.osd.2.smithi105.stderr:Reactor stalled for 473 ms on shard 0. Backtrace: 0x45d4d 0x442cc6e9 0x43fe48bb 0x43ffefad 0x43fff454 0x43fff6e8 0x43fff9b7 0x12cef 0xfaa62 0x3c0fc762 0x3c1001f6 0x3c103d6d 0x3c1049de 0x3c1050ac 0x3c0f8e37 0x3c0f931b 0x3c0f98c0 0x12cef 0x4eafe 0x21ea4 0x3bfefe00 0x36ae1216 0x38115705 0x37f70dc9 0x37f943e8 0x37f94d60 0x37f95588 0x37f95c5b 0x380acbcd 0x380ad0c0 0x43fc2c02 0x4401fd85 0x44219ea7 0x4421bebe 0x43b5c2cf 0x43b60bc1 0x361d2ca4 0x3ad84 0x35e7229d 2023-01-24T14:47:21.012 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6d64f6 0xffffffffac6dbbad 0xffffffffac4f10f6 0xffffffffac4f2c0a 0xffffffffac4f2efb 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:21.556 INFO:tasks.ceph.osd.2.smithi105.stderr:Reactor stalled for 1018 ms on shard 0. Backtrace: 0x45d4d 0x442cc6e9 0x43fe48bb 0x43ffefad 0x43fff454 0x43fff6e8 0x43fff9b7 0x12cef 0xfaa62 0x3c0fc762 0x3c1001f6 0x3c103d6d 0x3c1049de 0x3c1050ac 0x3c0f8e37 0x3c0f931b 0x3c0f98c0 0x12cef 0x4eafe 0x21ea4 0x3bfefe00 0x36ae1216 0x38115705 0x37f70dc9 0x37f943e8 0x37f94d60 0x37f95588 0x37f95c5b 0x380acbcd 0x380ad0c0 0x43fc2c02 0x4401fd85 0x44219ea7 0x4421bebe 0x43b5c2cf 0x43b60bc1 0x361d2ca4 0x3ad84 0x35e7229d 2023-01-24T14:47:21.556 INFO:tasks.ceph.osd.2.smithi105.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffac6d64e7 0xffffffffac6dbbad 0xffffffffac4f10f6 0xffffffffac4f2c0a 0xffffffffac4f2efb 0xffffffffac4043ab 0xfffffffface000a9 2023-01-24T14:47:21.980 INFO:tasks.ceph.osd.2.smithi105.stderr: 0# gsignal in /lib64/libc.so.6 2023-01-24T14:47:21.980 INFO:tasks.ceph.osd.2.smithi105.stderr: 1# abort in /lib64/libc.so.6 2023-01-24T14:47:21.981 INFO:tasks.ceph.osd.2.smithi105.stderr: 2# ceph::__ceph_assert_fail(char const*, char const*, int, char const*) in ceph-osd 2023-01-24T14:47:21.981 INFO:tasks.ceph.osd.2.smithi105.stderr: 3# crimson::osd::PG::is_degraded_or_backfilling_object(hobject_t const&) const in ceph-osd 2023-01-24T14:47:21.981 INFO:tasks.ceph.osd.2.smithi105.stderr: 4# crimson::osd::CommonClientRequest::do_recover_missing(boost::intrusive_ptr&, hobject_t const&) in ceph-osd 2023-01-24T14:47:21.981 INFO:tasks.ceph.osd.2.smithi105.stderr: 5# 0x00005592A4C72DCA in ceph-osd 2023-01-24T14:47:21.981 INFO:tasks.ceph.osd.2.smithi105.stderr: 6# 0x00005592A4C963E9 in ceph-osd 2023-01-24T14:47:21.982 INFO:tasks.ceph.osd.2.smithi105.stderr: 7# 0x00005592A4C96D61 in ceph-osd 2023-01-24T14:47:21.982 INFO:tasks.ceph.osd.2.smithi105.stderr: 8# 0x00005592A4C97589 in ceph-osd 2023-01-24T14:47:21.982 INFO:tasks.ceph.osd.2.smithi105.stderr: 9# 0x00005592A4C97C5C in ceph-osd 2023-01-24T14:47:21.982 INFO:tasks.ceph.osd.2.smithi105.stderr:10# auto seastar::internal::future_invoke > ()>&, seastar::internal::monostate>(seastar::noncopyable_function > ()>&, seastar::internal::monostate&&) in ceph-osd 2023-01-24T14:47:21.982 INFO:tasks.ceph.osd.2.smithi105.stderr:11# seastar::continuation, seastar::noncopyable_function > ()>, seastar::future::then_impl_nrvo > ()>, crimson::interruptible::interruptible_future_detail > >(seastar::noncopyable_function > ()>&&)::{lambda(seastar::internal::promise_base_with_type&&, seastar::noncopyable_function > ()>&, seastar::future_state&&)#1}, void>::run_and_dispose() in ceph-osd 2023-01-24T14:47:21.983 INFO:tasks.ceph.osd.2.smithi105.stderr:12# 0x00005592B0CC4C03 in ceph-osd 2023-01-24T14:47:21.983 INFO:tasks.ceph.osd.2.smithi105.stderr:13# 0x00005592B0D21D86 in ceph-osd 2023-01-24T14:47:21.983 INFO:tasks.ceph.osd.2.smithi105.stderr:14# 0x00005592B0F1BEA8 in ceph-osd 2023-01-24T14:47:21.983 INFO:tasks.ceph.osd.2.smithi105.stderr:15# 0x00005592B0F1DEBF in ceph-osd 2023-01-24T14:47:21.983 INFO:tasks.ceph.osd.2.smithi105.stderr:16# 0x00005592B085E2D0 in ceph-osd 2023-01-24T14:47:21.984 INFO:tasks.ceph.osd.2.smithi105.stderr:17# 0x00005592B0862BC2 in ceph-osd 2023-01-24T14:47:21.984 INFO:tasks.ceph.osd.2.smithi105.stderr:18# main in ceph-osd 2023-01-24T14:47:21.984 INFO:tasks.ceph.osd.2.smithi105.stderr:19# __libc_start_main in /lib64/libc.so.6 2023-01-24T14:47:21.984 INFO:tasks.ceph.osd.2.smithi105.stderr:20# _start in ceph-osd 2023-01-24T14:47:21.985 INFO:tasks.ceph.osd.2.smithi105.stderr:Content of /proc/self/maps: 2023-01-24T14:47:21.985 INFO:tasks.ceph.osd.2.smithi105.stderr:7fff7000-8fff7000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.985 INFO:tasks.ceph.osd.2.smithi105.stderr:8fff7000-2008fff7000 ---p 00000000 00:00 0 2023-01-24T14:47:21.985 INFO:tasks.ceph.osd.2.smithi105.stderr:2008fff7000-10007fff8000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.985 INFO:tasks.ceph.osd.2.smithi105.stderr:55926cd02000-5592a2b69000 r--p 00000000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:47:21.985 INFO:tasks.ceph.osd.2.smithi105.stderr:5592a2b69000-5592b21fb000 r-xp 35e67000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:47:21.986 INFO:tasks.ceph.osd.2.smithi105.stderr:5592b21fb000-5592b4933000 r--p 454f9000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:47:21.986 INFO:tasks.ceph.osd.2.smithi105.stderr:5592b4934000-5592b49e1000 r--p 47c31000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:47:21.986 INFO:tasks.ceph.osd.2.smithi105.stderr:5592b49e1000-5592ef1b9000 rw-p 47cde000 08:01 98451 /usr/bin/ceph-osd 2023-01-24T14:47:21.986 INFO:tasks.ceph.osd.2.smithi105.stderr:5592ef1b9000-5592ef228000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.986 INFO:tasks.ceph.osd.2.smithi105.stderr:600000000000-602000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.987 INFO:tasks.ceph.osd.2.smithi105.stderr:602000000000-6020000f0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.987 INFO:tasks.ceph.osd.2.smithi105.stderr:6020000f0000-602e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.987 INFO:tasks.ceph.osd.2.smithi105.stderr:602e00000000-602e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.987 INFO:tasks.ceph.osd.2.smithi105.stderr:602e00010000-603000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.987 INFO:tasks.ceph.osd.2.smithi105.stderr:603000000000-603001680000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.988 INFO:tasks.ceph.osd.2.smithi105.stderr:603001680000-603e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.988 INFO:tasks.ceph.osd.2.smithi105.stderr:603e00000000-603e001c0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.988 INFO:tasks.ceph.osd.2.smithi105.stderr:603e001c0000-604000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.988 INFO:tasks.ceph.osd.2.smithi105.stderr:604000000000-604000990000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.988 INFO:tasks.ceph.osd.2.smithi105.stderr:604000990000-604e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.988 INFO:tasks.ceph.osd.2.smithi105.stderr:604e00000000-604e00090000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.989 INFO:tasks.ceph.osd.2.smithi105.stderr:604e00090000-606000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.989 INFO:tasks.ceph.osd.2.smithi105.stderr:606000000000-6060015c0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.989 INFO:tasks.ceph.osd.2.smithi105.stderr:6060015c0000-606e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.989 INFO:tasks.ceph.osd.2.smithi105.stderr:606e00000000-606e000e0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.989 INFO:tasks.ceph.osd.2.smithi105.stderr:606e000e0000-607000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.990 INFO:tasks.ceph.osd.2.smithi105.stderr:607000000000-607000450000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.990 INFO:tasks.ceph.osd.2.smithi105.stderr:607000450000-607e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.990 INFO:tasks.ceph.osd.2.smithi105.stderr:607e00000000-607e00020000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.990 INFO:tasks.ceph.osd.2.smithi105.stderr:607e00020000-608000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.990 INFO:tasks.ceph.osd.2.smithi105.stderr:608000000000-608004d30000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.991 INFO:tasks.ceph.osd.2.smithi105.stderr:608004d30000-608e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.991 INFO:tasks.ceph.osd.2.smithi105.stderr:608e00000000-608e00240000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.991 INFO:tasks.ceph.osd.2.smithi105.stderr:608e00240000-60a000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.991 INFO:tasks.ceph.osd.2.smithi105.stderr:60a000000000-60a000030000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.991 INFO:tasks.ceph.osd.2.smithi105.stderr:60a000030000-60ae00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.992 INFO:tasks.ceph.osd.2.smithi105.stderr:60ae00000000-60ae00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.992 INFO:tasks.ceph.osd.2.smithi105.stderr:60ae00010000-60b000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.992 INFO:tasks.ceph.osd.2.smithi105.stderr:60b000000000-60b000f30000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.992 INFO:tasks.ceph.osd.2.smithi105.stderr:60b000f30000-60be00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.993 INFO:tasks.ceph.osd.2.smithi105.stderr:60be00000000-60be00050000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.995 INFO:tasks.ceph.osd.2.smithi105.stderr:60be00050000-60c000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.995 INFO:tasks.ceph.osd.2.smithi105.stderr:60c000000000-60c001da0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.996 INFO:tasks.ceph.osd.2.smithi105.stderr:60c001da0000-60ce00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.996 INFO:tasks.ceph.osd.2.smithi105.stderr:60ce00000000-60ce000a0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.996 INFO:tasks.ceph.osd.2.smithi105.stderr:60ce000a0000-60d000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.996 INFO:tasks.ceph.osd.2.smithi105.stderr:60d000000000-60d000060000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.996 INFO:tasks.ceph.osd.2.smithi105.stderr:60d000060000-60de00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.997 INFO:tasks.ceph.osd.2.smithi105.stderr:60de00000000-60de00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.997 INFO:tasks.ceph.osd.2.smithi105.stderr:60de00010000-60e000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.997 INFO:tasks.ceph.osd.2.smithi105.stderr:60e000000000-60e0003c0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.997 INFO:tasks.ceph.osd.2.smithi105.stderr:60e0003c0000-60ee00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.997 INFO:tasks.ceph.osd.2.smithi105.stderr:60ee00000000-60ee00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.998 INFO:tasks.ceph.osd.2.smithi105.stderr:60ee00010000-60f000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.998 INFO:tasks.ceph.osd.2.smithi105.stderr:60f000000000-60f0000d0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.998 INFO:tasks.ceph.osd.2.smithi105.stderr:60f0000d0000-60fe00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.998 INFO:tasks.ceph.osd.2.smithi105.stderr:60fe00000000-60fe00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.999 INFO:tasks.ceph.osd.2.smithi105.stderr:60fe00010000-610000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.999 INFO:tasks.ceph.osd.2.smithi105.stderr:610000000000-6100000e0000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.999 INFO:tasks.ceph.osd.2.smithi105.stderr:6100000e0000-610e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.999 INFO:tasks.ceph.osd.2.smithi105.stderr:610e00000000-610e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:21.999 INFO:tasks.ceph.osd.2.smithi105.stderr:610e00010000-611000000000 ---p 00000000 00:00 0 2023-01-24T14:47:21.999 INFO:tasks.ceph.osd.2.smithi105.stderr:611000000000-611000700000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.000 INFO:tasks.ceph.osd.2.smithi105.stderr:611000700000-611e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.000 INFO:tasks.ceph.osd.2.smithi105.stderr:611e00000000-611e00020000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.000 INFO:tasks.ceph.osd.2.smithi105.stderr:611e00020000-612000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.001 INFO:tasks.ceph.osd.2.smithi105.stderr:612000000000-6120000e0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.001 INFO:tasks.ceph.osd.2.smithi105.stderr:6120000e0000-612e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.001 INFO:tasks.ceph.osd.2.smithi105.stderr:612e00000000-612e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.001 INFO:tasks.ceph.osd.2.smithi105.stderr:612e00010000-613000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.002 INFO:tasks.ceph.osd.2.smithi105.stderr:613000000000-6130002b0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.002 INFO:tasks.ceph.osd.2.smithi105.stderr:6130002b0000-613e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.002 INFO:tasks.ceph.osd.2.smithi105.stderr:613e00000000-613e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.002 INFO:tasks.ceph.osd.2.smithi105.stderr:613e00010000-614000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.002 INFO:tasks.ceph.osd.2.smithi105.stderr:614000000000-614000190000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.003 INFO:tasks.ceph.osd.2.smithi105.stderr:614000190000-614e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.004 INFO:tasks.ceph.osd.2.smithi105.stderr:614e00000000-614e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.004 INFO:tasks.ceph.osd.2.smithi105.stderr:614e00010000-615000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.004 INFO:tasks.ceph.osd.2.smithi105.stderr:615000000000-6150001a0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.004 INFO:tasks.ceph.osd.2.smithi105.stderr:6150001a0000-615e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.004 INFO:tasks.ceph.osd.2.smithi105.stderr:615e00000000-615e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.005 INFO:tasks.ceph.osd.2.smithi105.stderr:615e00010000-616000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.005 INFO:tasks.ceph.osd.2.smithi105.stderr:616000000000-616000a50000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.005 INFO:tasks.ceph.osd.2.smithi105.stderr:616000a50000-616e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.005 INFO:tasks.ceph.osd.2.smithi105.stderr:616e00000000-616e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.005 INFO:tasks.ceph.osd.2.smithi105.stderr:616e00010000-617000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.006 INFO:tasks.ceph.osd.2.smithi105.stderr:617000000000-617000560000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.006 INFO:tasks.ceph.osd.2.smithi105.stderr:617000560000-617e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.006 INFO:tasks.ceph.osd.2.smithi105.stderr:617e00000000-617e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.006 INFO:tasks.ceph.osd.2.smithi105.stderr:617e00010000-618000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.006 INFO:tasks.ceph.osd.2.smithi105.stderr:618000000000-6180000f0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.007 INFO:tasks.ceph.osd.2.smithi105.stderr:6180000f0000-618e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.007 INFO:tasks.ceph.osd.2.smithi105.stderr:618e00000000-618e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.007 INFO:tasks.ceph.osd.2.smithi105.stderr:618e00010000-619000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.007 INFO:tasks.ceph.osd.2.smithi105.stderr:619000000000-619000e10000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.007 INFO:tasks.ceph.osd.2.smithi105.stderr:619000e10000-619e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.008 INFO:tasks.ceph.osd.2.smithi105.stderr:619e00000000-619e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.008 INFO:tasks.ceph.osd.2.smithi105.stderr:619e00010000-61a000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.008 INFO:tasks.ceph.osd.2.smithi105.stderr:61a000000000-61a000110000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.008 INFO:tasks.ceph.osd.2.smithi105.stderr:61a000110000-61ae00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.009 INFO:tasks.ceph.osd.2.smithi105.stderr:61ae00000000-61ae00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.009 INFO:tasks.ceph.osd.2.smithi105.stderr:61ae00010000-61b000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.010 INFO:tasks.ceph.osd.2.smithi105.stderr:61b000000000-61b000060000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.010 INFO:tasks.ceph.osd.2.smithi105.stderr:61b000060000-61be00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.010 INFO:tasks.ceph.osd.2.smithi105.stderr:61be00000000-61be00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.011 INFO:tasks.ceph.osd.2.smithi105.stderr:61be00010000-61c000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.011 INFO:tasks.ceph.osd.2.smithi105.stderr:61c000000000-61c0000d0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.011 INFO:tasks.ceph.osd.2.smithi105.stderr:61c0000d0000-61ce00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.012 INFO:tasks.ceph.osd.2.smithi105.stderr:61ce00000000-61ce00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.012 INFO:tasks.ceph.osd.2.smithi105.stderr:61ce00010000-61d000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.013 INFO:tasks.ceph.osd.2.smithi105.stderr:61d000000000-61d0001d0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.013 INFO:tasks.ceph.osd.2.smithi105.stderr:61d0001d0000-61de00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.013 INFO:tasks.ceph.osd.2.smithi105.stderr:61de00000000-61de00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.014 INFO:tasks.ceph.osd.2.smithi105.stderr:61de00010000-61e000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.014 INFO:tasks.ceph.osd.2.smithi105.stderr:61e000000000-61e0000b0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.014 INFO:tasks.ceph.osd.2.smithi105.stderr:61e0000b0000-61ee00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.015 INFO:tasks.ceph.osd.2.smithi105.stderr:61ee00000000-61ee00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.015 INFO:tasks.ceph.osd.2.smithi105.stderr:61ee00010000-61f000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.016 INFO:tasks.ceph.osd.2.smithi105.stderr:61f000000000-61f000070000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.016 INFO:tasks.ceph.osd.2.smithi105.stderr:61f000070000-61fe00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.016 INFO:tasks.ceph.osd.2.smithi105.stderr:61fe00000000-61fe00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.017 INFO:tasks.ceph.osd.2.smithi105.stderr:61fe00010000-620000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.017 INFO:tasks.ceph.osd.2.smithi105.stderr:620000000000-620000090000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.017 INFO:tasks.ceph.osd.2.smithi105.stderr:620000090000-620e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.018 INFO:tasks.ceph.osd.2.smithi105.stderr:620e00000000-620e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.018 INFO:tasks.ceph.osd.2.smithi105.stderr:620e00010000-621000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.018 INFO:tasks.ceph.osd.2.smithi105.stderr:621000000000-621003910000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.019 INFO:tasks.ceph.osd.2.smithi105.stderr:621003910000-621e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.019 INFO:tasks.ceph.osd.2.smithi105.stderr:621e00000000-621e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.020 INFO:tasks.ceph.osd.2.smithi105.stderr:621e00010000-622000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.020 INFO:tasks.ceph.osd.2.smithi105.stderr:622000000000-6220002f0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.020 INFO:tasks.ceph.osd.2.smithi105.stderr:6220002f0000-622e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.021 INFO:tasks.ceph.osd.2.smithi105.stderr:622e00000000-622e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.021 INFO:tasks.ceph.osd.2.smithi105.stderr:622e00010000-623000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.021 INFO:tasks.ceph.osd.2.smithi105.stderr:623000000000-623000070000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.022 INFO:tasks.ceph.osd.2.smithi105.stderr:623000070000-623e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.022 INFO:tasks.ceph.osd.2.smithi105.stderr:623e00000000-623e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.022 INFO:tasks.ceph.osd.2.smithi105.stderr:623e00010000-624000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.023 INFO:tasks.ceph.osd.2.smithi105.stderr:624000000000-624001080000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.023 INFO:tasks.ceph.osd.2.smithi105.stderr:624001080000-624e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.024 INFO:tasks.ceph.osd.2.smithi105.stderr:624e00000000-624e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.024 INFO:tasks.ceph.osd.2.smithi105.stderr:624e00010000-625000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.024 INFO:tasks.ceph.osd.2.smithi105.stderr:625000000000-625003d70000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.025 INFO:tasks.ceph.osd.2.smithi105.stderr:625003d70000-625e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.025 INFO:tasks.ceph.osd.2.smithi105.stderr:625e00000000-625e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.025 INFO:tasks.ceph.osd.2.smithi105.stderr:625e00010000-626000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.026 INFO:tasks.ceph.osd.2.smithi105.stderr:626000000000-626000120000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.026 INFO:tasks.ceph.osd.2.smithi105.stderr:626000120000-626e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.026 INFO:tasks.ceph.osd.2.smithi105.stderr:626e00000000-626e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.027 INFO:tasks.ceph.osd.2.smithi105.stderr:626e00010000-627000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.027 INFO:tasks.ceph.osd.2.smithi105.stderr:627000000000-627000050000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.028 INFO:tasks.ceph.osd.2.smithi105.stderr:627000050000-627e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.028 INFO:tasks.ceph.osd.2.smithi105.stderr:627e00000000-627e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.028 INFO:tasks.ceph.osd.2.smithi105.stderr:627e00010000-628000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.029 INFO:tasks.ceph.osd.2.smithi105.stderr:628000000000-628000020000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.029 INFO:tasks.ceph.osd.2.smithi105.stderr:628000020000-628e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.029 INFO:tasks.ceph.osd.2.smithi105.stderr:628e00000000-628e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.030 INFO:tasks.ceph.osd.2.smithi105.stderr:628e00010000-629000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.030 INFO:tasks.ceph.osd.2.smithi105.stderr:629000000000-629000150000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.030 INFO:tasks.ceph.osd.2.smithi105.stderr:629000150000-629e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.031 INFO:tasks.ceph.osd.2.smithi105.stderr:629e00000000-629e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.031 INFO:tasks.ceph.osd.2.smithi105.stderr:629e00010000-62a000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.031 INFO:tasks.ceph.osd.2.smithi105.stderr:62a000000000-62a000100000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.032 INFO:tasks.ceph.osd.2.smithi105.stderr:62a000100000-62ae00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.032 INFO:tasks.ceph.osd.2.smithi105.stderr:62ae00000000-62ae00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.033 INFO:tasks.ceph.osd.2.smithi105.stderr:62ae00010000-62b000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.033 INFO:tasks.ceph.osd.2.smithi105.stderr:62b000000000-62b000030000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.033 INFO:tasks.ceph.osd.2.smithi105.stderr:62b000030000-62be00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.034 INFO:tasks.ceph.osd.2.smithi105.stderr:62be00000000-62be00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.034 INFO:tasks.ceph.osd.2.smithi105.stderr:62be00010000-62c000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.034 INFO:tasks.ceph.osd.2.smithi105.stderr:62c000000000-62c000010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.035 INFO:tasks.ceph.osd.2.smithi105.stderr:62c000010000-62ce00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.035 INFO:tasks.ceph.osd.2.smithi105.stderr:62ce00000000-62ce00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.035 INFO:tasks.ceph.osd.2.smithi105.stderr:62ce00010000-62d000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.036 INFO:tasks.ceph.osd.2.smithi105.stderr:62d000000000-62d000230000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.036 INFO:tasks.ceph.osd.2.smithi105.stderr:62d000230000-62de00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.036 INFO:tasks.ceph.osd.2.smithi105.stderr:62de00000000-62de00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.037 INFO:tasks.ceph.osd.2.smithi105.stderr:62de00010000-62e000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.037 INFO:tasks.ceph.osd.2.smithi105.stderr:62e000000000-62e000060000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.037 INFO:tasks.ceph.osd.2.smithi105.stderr:62e000060000-62ee00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.038 INFO:tasks.ceph.osd.2.smithi105.stderr:62ee00000000-62ee00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.038 INFO:tasks.ceph.osd.2.smithi105.stderr:62ee00010000-62f000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.038 INFO:tasks.ceph.osd.2.smithi105.stderr:62f000000000-62f0000c0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.039 INFO:tasks.ceph.osd.2.smithi105.stderr:62f0000c0000-62fe00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.039 INFO:tasks.ceph.osd.2.smithi105.stderr:62fe00000000-62fe00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.039 INFO:tasks.ceph.osd.2.smithi105.stderr:62fe00010000-630000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.040 INFO:tasks.ceph.osd.2.smithi105.stderr:630000000000-630000090000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.040 INFO:tasks.ceph.osd.2.smithi105.stderr:630000090000-630e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.040 INFO:tasks.ceph.osd.2.smithi105.stderr:630e00000000-630e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.041 INFO:tasks.ceph.osd.2.smithi105.stderr:630e00010000-631000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.041 INFO:tasks.ceph.osd.2.smithi105.stderr:631000000000-631003810000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.042 INFO:tasks.ceph.osd.2.smithi105.stderr:631003810000-631e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.042 INFO:tasks.ceph.osd.2.smithi105.stderr:631e00000000-631e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.042 INFO:tasks.ceph.osd.2.smithi105.stderr:631e00010000-632000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.043 INFO:tasks.ceph.osd.2.smithi105.stderr:632000000000-632000020000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.043 INFO:tasks.ceph.osd.2.smithi105.stderr:632000020000-632e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.043 INFO:tasks.ceph.osd.2.smithi105.stderr:632e00000000-632e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.044 INFO:tasks.ceph.osd.2.smithi105.stderr:632e00010000-633000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.044 INFO:tasks.ceph.osd.2.smithi105.stderr:633000000000-633000070000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.045 INFO:tasks.ceph.osd.2.smithi105.stderr:633000070000-633e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.045 INFO:tasks.ceph.osd.2.smithi105.stderr:633e00000000-633e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.045 INFO:tasks.ceph.osd.2.smithi105.stderr:633e00010000-634000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.046 INFO:tasks.ceph.osd.2.smithi105.stderr:634000000000-634000080000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.046 INFO:tasks.ceph.osd.2.smithi105.stderr:634000080000-634e00000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.046 INFO:tasks.ceph.osd.2.smithi105.stderr:634e00000000-634e00010000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.047 INFO:tasks.ceph.osd.2.smithi105.stderr:634e00010000-640000000000 ---p 00000000 00:00 0 2023-01-24T14:47:22.047 INFO:tasks.ceph.osd.2.smithi105.stderr:640000000000-640000003000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.047 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e47a1c000-7f6e48728000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.047 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e49613000-7f6e4b6da000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.048 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e4b6fc000-7f6e4cc50000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.048 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e4cc59000-7f6e4db4b000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.048 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e4db5d000-7f6e4f4f8000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.048 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e4f50d000-7f6e5184d000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.048 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e51890000-7f6e51992000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.049 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e526b2000-7f6e52c1d000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.049 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e52c39000-7f6e5328e000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.049 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5329f000-7f6e53889000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.049 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e538a9000-7f6e538cb000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.049 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e538e4000-7f6e549f3000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.050 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e549f3000-7f6e54d17000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.050 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e54d17000-7f6e54d18000 ---p 00000000 00:00 0 2023-01-24T14:47:22.050 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e54d18000-7f6e55518000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.050 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e55525000-7f6e55ee4000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.050 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e55ff0000-7f6e560f2000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.051 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e56105000-7f6e56125000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.051 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e56168000-7f6e56178000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.051 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e56459000-7f6e56522000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.051 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e56522000-7f6e56523000 ---p 00000000 00:00 0 2023-01-24T14:47:22.051 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e56523000-7f6e56d23000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.052 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e56d23000-7f6e56d24000 ---p 00000000 00:00 0 2023-01-24T14:47:22.052 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e56d24000-7f6e57524000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.052 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e57524000-7f6e57525000 ---p 00000000 00:00 0 2023-01-24T14:47:22.052 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e57525000-7f6e57d25000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.052 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e57d2c000-7f6e57d5c000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.053 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e57d5f000-7f6e57d7f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.053 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e57da0000-7f6e57e15000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.053 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e57e34000-7f6e57efe000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.053 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e57f15000-7f6e581ff000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.053 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e58214000-7f6e58d06000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.053 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e58d16000-7f6e59029000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.054 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5903d000-7f6e59103000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.054 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59109000-7f6e591ae000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.054 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e591be000-7f6e591f0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.054 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e591ff000-7f6e59274000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.054 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5927e000-7f6e592d1000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.055 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e592db000-7f6e5933e000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.055 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59340000-7f6e593a5000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.055 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e593c6000-7f6e59408000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.055 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59417000-7f6e59447000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.055 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5944b000-7f6e5948d000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.056 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5949b000-7f6e59500000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.056 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59514000-7f6e5959a000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.056 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5959d000-7f6e59620000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.056 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5962d000-7f6e5963d000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.056 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59642000-7f6e596a5000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.056 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e596a6000-7f6e596b6000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.057 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e596b8000-7f6e596f8000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.057 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e596fb000-7f6e5971b000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.057 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5973b000-7f6e5988a000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.057 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e598aa000-7f6e59920000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.058 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59921000-7f6e59943000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.058 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5995c000-7f6e59c04000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.058 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59c25000-7f6e59c87000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.058 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59c89000-7f6e59d0d000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.058 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e59d2b000-7f6e5a312000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.059 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5a312000-7f6e5a461000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.059 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5a461000-7f6e5a52a000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.059 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5a52a000-7f6e5a52b000 ---p 00000000 00:00 0 2023-01-24T14:47:22.059 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5a52b000-7f6e5ad2c000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.059 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ad3a000-7f6e5ae23000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.060 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ae25000-7f6e5b0c0000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.060 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b0c1000-7f6e5b206000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.060 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b209000-7f6e5b249000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.060 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b24c000-7f6e5b459000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.060 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b463000-7f6e5b584000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.061 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b592000-7f6e5b5b2000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.061 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b5b3000-7f6e5b6ce000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.061 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b6cf000-7f6e5b910000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.061 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5b91a000-7f6e5ba23000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.061 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ba25000-7f6e5ba55000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.061 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ba59000-7f6e5babc000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.062 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5babe000-7f6e5bb61000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.062 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bb62000-7f6e5bb82000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.062 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bb84000-7f6e5bca6000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.062 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bca7000-7f6e5bcc7000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.063 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bcc9000-7f6e5bd4c000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.063 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bd55000-7f6e5bd75000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.063 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bd78000-7f6e5be3e000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.063 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5be4a000-7f6e5be9d000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.063 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bea9000-7f6e5bf0c000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.064 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bf0e000-7f6e5bf2e000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.064 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bf2f000-7f6e5bf91000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.064 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5bf99000-7f6e5c0fd000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.064 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c101000-7f6e5c143000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.064 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c145000-7f6e5c175000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.064 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c178000-7f6e5c1e8000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.065 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c1e8000-7f6e5c1f0000 r--p 00000000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:47:22.065 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c1f0000-7f6e5c1f5000 r-xp 00008000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:47:22.065 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c1f5000-7f6e5c1f7000 r--p 0000d000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:47:22.065 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c1f7000-7f6e5c1f8000 ---p 0000f000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:47:22.065 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c1f8000-7f6e5c1f9000 r--p 0000f000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:47:22.066 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c1f9000-7f6e5c1fa000 rw-p 00010000 08:01 525535 /usr/lib64/ceph/compressor/libceph_snappy.so.2.0.0 2023-01-24T14:47:22.066 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c1fa000-7f6e5c268000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.066 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c268000-7f6e5c269000 ---p 00000000 00:00 0 2023-01-24T14:47:22.066 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5c269000-7f6e5ca7f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.066 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ca7f000-7f6e5ca80000 ---p 00000000 00:00 0 2023-01-24T14:47:22.067 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ca80000-7f6e5d296000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.067 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5d296000-7f6e5d297000 ---p 00000000 00:00 0 2023-01-24T14:47:22.067 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5d297000-7f6e5daad000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.067 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5daad000-7f6e5daae000 ---p 00000000 00:00 0 2023-01-24T14:47:22.067 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5daae000-7f6e5e2ae000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.068 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5e2b9000-7f6e5e2d9000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.068 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5e2e7000-7f6e5e37a000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.068 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5e385000-7f6e5e3ed000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.068 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5e3ed000-7f6e5e3fb000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.068 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5e3fb000-7f6e5e3fc000 ---p 00000000 00:00 0 2023-01-24T14:47:22.068 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5e3fc000-7f6e5ebfc000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.069 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ebfc000-7f6e5ec0d000 rw-s 00000000 00:12 435071 /[aio] (deleted) 2023-01-24T14:47:22.069 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ec0d000-7f6e5ec23000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.069 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ec23000-7f6e5ec24000 ---p 00000000 00:00 0 2023-01-24T14:47:22.069 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5ec24000-7f6e5f424000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.069 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5f424000-7f6e5f435000 rw-s 00000000 00:12 433843 /[aio] (deleted) 2023-01-24T14:47:22.070 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5f435000-7f6e5f44b000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.070 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5f44b000-7f6e5f44c000 ---p 00000000 00:00 0 2023-01-24T14:47:22.070 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5f44c000-7f6e5fc62000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.070 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5fc62000-7f6e5fc63000 ---p 00000000 00:00 0 2023-01-24T14:47:22.070 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e5fc63000-7f6e60479000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.071 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e60479000-7f6e6047a000 ---p 00000000 00:00 0 2023-01-24T14:47:22.071 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6047a000-7f6e60c90000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.071 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e60c90000-7f6e60c91000 ---p 00000000 00:00 0 2023-01-24T14:47:22.071 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e60c91000-7f6e614a7000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.071 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e614a7000-7f6e614a8000 ---p 00000000 00:00 0 2023-01-24T14:47:22.071 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e614a8000-7f6e61cbe000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.072 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e61cbe000-7f6e61cbf000 ---p 00000000 00:00 0 2023-01-24T14:47:22.072 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e61cbf000-7f6e624d5000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.072 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e624d5000-7f6e624d6000 ---p 00000000 00:00 0 2023-01-24T14:47:22.072 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e624d6000-7f6e6375f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.072 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6375f000-7f6e63760000 ---p 00000000 00:00 0 2023-01-24T14:47:22.073 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e63760000-7f6e63f70000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.073 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e63f73000-7f6e63fb3000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.073 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e63fb3000-7f6e63fc2000 r--p 00000000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:47:22.073 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e63fc2000-7f6e63fef000 r-xp 0000f000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:47:22.073 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e63fef000-7f6e63ffe000 r--p 0003c000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:47:22.073 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e63ffe000-7f6e63fff000 ---p 0004b000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:47:22.074 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e63fff000-7f6e64001000 r--p 0004b000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:47:22.074 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64001000-7f6e64002000 rw-p 0004d000 08:01 525612 /usr/lib64/rados-classes/libcls_rgw_gc.so.1.0.0 2023-01-24T14:47:22.074 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64002000-7f6e64009000 r--p 00000000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:47:22.075 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64009000-7f6e64016000 r-xp 00007000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:47:22.075 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64016000-7f6e6401a000 r--p 00014000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:47:22.075 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6401a000-7f6e6401b000 ---p 00018000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:47:22.076 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6401b000-7f6e6401c000 r--p 00018000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:47:22.076 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6401c000-7f6e6401d000 rw-p 00019000 08:01 525585 /usr/lib64/rados-classes/libcls_lock.so.1.0.0 2023-01-24T14:47:22.076 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6401d000-7f6e64028000 r--p 00000000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:47:22.077 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64028000-7f6e64035000 r-xp 0000b000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:47:22.077 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64035000-7f6e6403a000 r--p 00018000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:47:22.077 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6403a000-7f6e6403c000 r--p 0001c000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:47:22.077 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6403c000-7f6e6403d000 rw-p 0001e000 08:01 525567 /usr/lib64/rados-classes/libcls_cas.so.1.0.0 2023-01-24T14:47:22.077 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6403d000-7f6e6404c000 r--p 00000000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:47:22.078 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6404c000-7f6e64073000 r-xp 0000f000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:47:22.078 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64073000-7f6e64080000 r--p 00036000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:47:22.078 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64080000-7f6e64081000 ---p 00043000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:47:22.079 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64081000-7f6e64083000 r--p 00043000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:47:22.079 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64083000-7f6e64084000 rw-p 00045000 08:01 525600 /usr/lib64/rados-classes/libcls_queue.so.1.0.0 2023-01-24T14:47:22.079 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64084000-7f6e6408b000 r--p 00000000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:47:22.079 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6408b000-7f6e64093000 r-xp 00007000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:47:22.079 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64093000-7f6e64096000 r--p 0000f000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:47:22.079 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64096000-7f6e64097000 ---p 00012000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:47:22.080 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64097000-7f6e64098000 r--p 00012000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:47:22.080 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64098000-7f6e64099000 rw-p 00013000 08:01 525588 /usr/lib64/rados-classes/libcls_log.so.1.0.0 2023-01-24T14:47:22.080 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64099000-7f6e640a0000 r--p 00000000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:47:22.080 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640a0000-7f6e640ab000 r-xp 00007000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:47:22.080 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640ab000-7f6e640b1000 r--p 00012000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:47:22.081 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640b1000-7f6e640b2000 ---p 00018000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:47:22.081 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640b2000-7f6e640b3000 r--p 00018000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:47:22.081 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640b3000-7f6e640b4000 rw-p 00019000 08:01 525573 /usr/lib64/rados-classes/libcls_cmpomap.so.1.0.0 2023-01-24T14:47:22.081 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640b4000-7f6e640c3000 r--p 00000000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:47:22.081 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640c3000-7f6e640ef000 r-xp 0000f000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:47:22.082 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640ef000-7f6e640fe000 r--p 0003b000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:47:22.082 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640fe000-7f6e640ff000 ---p 0004a000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:47:22.082 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e640ff000-7f6e64101000 r--p 0004a000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:47:22.082 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64101000-7f6e64102000 rw-p 0004c000 08:01 525564 /usr/lib64/rados-classes/libcls_2pc_queue.so.1.0.0 2023-01-24T14:47:22.082 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64102000-7f6e6410a000 r--p 00000000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:47:22.083 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6410a000-7f6e64133000 r-xp 00008000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:47:22.083 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64133000-7f6e64140000 r--p 00031000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:47:22.083 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64140000-7f6e64141000 ---p 0003e000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:47:22.083 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64141000-7f6e64142000 r--p 0003e000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:47:22.083 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64142000-7f6e64143000 rw-p 0003f000 08:01 525576 /usr/lib64/rados-classes/libcls_fifo.so.1.0.0 2023-01-24T14:47:22.083 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64143000-7f6e64150000 r--p 00000000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:47:22.084 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64150000-7f6e641a8000 r-xp 0000d000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:47:22.084 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e641a8000-7f6e641bf000 r--p 00065000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:47:22.084 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e641bf000-7f6e641c1000 r--p 0007b000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:47:22.084 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e641c1000-7f6e641c2000 rw-p 0007d000 08:01 525603 /usr/lib64/rados-classes/libcls_rbd.so.1.0.0 2023-01-24T14:47:22.084 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e641c2000-7f6e64274000 r--p 00000000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:47:22.085 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64274000-7f6e642ee000 r-xp 000b2000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:47:22.085 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e642ee000-7f6e6432e000 r--p 0012c000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:47:22.085 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6432e000-7f6e6432f000 ---p 0016c000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:47:22.085 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6432f000-7f6e64335000 r--p 0016c000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:47:22.085 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64335000-7f6e64336000 rw-p 00172000 08:01 525606 /usr/lib64/rados-classes/libcls_refcount.so.1.0.0 2023-01-24T14:47:22.086 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64336000-7f6e6433c000 r--p 00000000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:47:22.086 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6433c000-7f6e64343000 r-xp 00006000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:47:22.086 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64343000-7f6e64346000 r--p 0000d000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:47:22.086 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64346000-7f6e64347000 r--p 0000f000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:47:22.086 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64347000-7f6e64348000 rw-p 00010000 08:01 525621 /usr/lib64/rados-classes/libcls_timeindex.so.1.0.0 2023-01-24T14:47:22.087 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64348000-7f6e64350000 r--p 00000000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:47:22.087 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64350000-7f6e64360000 r-xp 00008000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:47:22.087 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64360000-7f6e64365000 r--p 00018000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:47:22.087 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64365000-7f6e64366000 ---p 0001d000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:47:22.087 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64366000-7f6e64367000 r--p 0001d000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:47:22.087 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64367000-7f6e64368000 rw-p 0001e000 08:01 525582 /usr/lib64/rados-classes/libcls_journal.so.1.0.0 2023-01-24T14:47:22.088 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64368000-7f6e6436e000 r--p 00000000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:47:22.088 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6436e000-7f6e64379000 r-xp 00006000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:47:22.088 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64379000-7f6e6437d000 r--p 00011000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:47:22.088 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6437d000-7f6e6437e000 r--p 00014000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:47:22.088 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6437e000-7f6e6437f000 rw-p 00015000 08:01 525624 /usr/lib64/rados-classes/libcls_user.so.1.0.0 2023-01-24T14:47:22.089 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6437f000-7f6e64386000 r--p 00000000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:47:22.089 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64386000-7f6e6438c000 r-xp 00007000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:47:22.089 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6438c000-7f6e6438f000 r--p 0000d000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:47:22.089 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6438f000-7f6e64390000 r--p 0000f000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:47:22.089 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64390000-7f6e64391000 rw-p 00010000 08:01 525570 /usr/lib64/rados-classes/libcls_cephfs.so.1.0.0 2023-01-24T14:47:22.090 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64391000-7f6e64444000 r--p 00000000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:47:22.090 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64444000-7f6e64512000 r-xp 000b3000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:47:22.090 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64512000-7f6e64568000 r--p 00181000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:47:22.090 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64568000-7f6e6456f000 r--p 001d6000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:47:22.090 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6456f000-7f6e64570000 rw-p 001dd000 08:01 525609 /usr/lib64/rados-classes/libcls_rgw.so.1.0.0 2023-01-24T14:47:22.090 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64570000-7f6e64576000 r--p 00000000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:47:22.091 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64576000-7f6e6457c000 r-xp 00006000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:47:22.091 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6457c000-7f6e6457f000 r--p 0000c000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:47:22.091 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6457f000-7f6e64580000 r--p 0000e000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:47:22.091 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64580000-7f6e64581000 rw-p 0000f000 08:01 525627 /usr/lib64/rados-classes/libcls_version.so.1.0.0 2023-01-24T14:47:22.091 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64581000-7f6e64594000 r-xp 00000000 08:01 98590 /usr/lib64/liboath.so.0.1.3 2023-01-24T14:47:22.091 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64594000-7f6e64794000 ---p 00013000 08:01 98590 /usr/lib64/liboath.so.0.1.3 2023-01-24T14:47:22.092 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64794000-7f6e64795000 r--p 00013000 08:01 98590 /usr/lib64/liboath.so.0.1.3 2023-01-24T14:47:22.092 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64795000-7f6e6479d000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.092 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6479d000-7f6e647a5000 r--p 00000000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:47:22.092 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647a5000-7f6e647b1000 r-xp 00008000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:47:22.092 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647b1000-7f6e647b5000 r--p 00014000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:47:22.093 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647b5000-7f6e647b6000 ---p 00018000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:47:22.093 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647b6000-7f6e647b7000 r--p 00018000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:47:22.093 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647b7000-7f6e647b8000 rw-p 00019000 08:01 525597 /usr/lib64/rados-classes/libcls_otp.so.1.0.0 2023-01-24T14:47:22.093 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647b8000-7f6e647be000 r--p 00000000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:47:22.093 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647be000-7f6e647c3000 r-xp 00006000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:47:22.094 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647c3000-7f6e647c5000 r--p 0000b000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:47:22.094 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647c5000-7f6e647c6000 ---p 0000d000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:47:22.094 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647c6000-7f6e647c7000 r--p 0000d000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:47:22.094 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647c7000-7f6e647c8000 rw-p 0000e000 08:01 525594 /usr/lib64/rados-classes/libcls_numops.so.1.0.0 2023-01-24T14:47:22.094 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647c8000-7f6e647cf000 r--p 00000000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:47:22.094 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647cf000-7f6e647d5000 r-xp 00007000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:47:22.095 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647d5000-7f6e647d7000 r--p 0000d000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:47:22.095 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647d7000-7f6e647d8000 ---p 0000f000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:47:22.095 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647d8000-7f6e647d9000 r--p 0000f000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:47:22.095 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647d9000-7f6e647da000 rw-p 00010000 08:01 525579 /usr/lib64/rados-classes/libcls_hello.so.1.0.0 2023-01-24T14:47:22.095 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647da000-7f6e647ec000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.096 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647ec000-7f6e647ed000 r--p 00000000 00:00 0 2023-01-24T14:47:22.096 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e647ed000-7f6e64853000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.096 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64853000-7f6e64854000 ---p 00000000 00:00 0 2023-01-24T14:47:22.097 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e64854000-7f6e65054000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.097 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e65054000-7f6e65065000 rw-s 00000000 00:12 437468 /[aio] (deleted) 2023-01-24T14:47:22.097 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e65065000-7f6e6509e000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.098 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6509e000-7f6e6509f000 ---p 00000000 00:00 0 2023-01-24T14:47:22.098 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6509f000-7f6e698e3000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.098 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e698e3000-7f6e698e7000 r-xp 00000000 08:01 13845 /usr/lib64/liburcu-common.so.6.0.0 2023-01-24T14:47:22.099 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e698e7000-7f6e69ae6000 ---p 00004000 08:01 13845 /usr/lib64/liburcu-common.so.6.0.0 2023-01-24T14:47:22.099 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69ae6000-7f6e69ae7000 r--p 00003000 08:01 13845 /usr/lib64/liburcu-common.so.6.0.0 2023-01-24T14:47:22.099 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69ae7000-7f6e69ae8000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.100 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69ae8000-7f6e69af1000 r-xp 00000000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:47:22.100 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69af1000-7f6e69cf0000 ---p 00009000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:47:22.100 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69cf0000-7f6e69cf1000 r--p 00008000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:47:22.100 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69cf1000-7f6e69cf2000 rw-p 00009000 08:01 13843 /usr/lib64/liburcu-cds.so.6.0.0 2023-01-24T14:47:22.101 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69cf2000-7f6e69cf9000 r-xp 00000000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:47:22.101 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69cf9000-7f6e69ef8000 ---p 00007000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:47:22.101 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69ef8000-7f6e69ef9000 r--p 00006000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:47:22.101 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69ef9000-7f6e69efa000 rw-p 00007000 08:01 13841 /usr/lib64/liburcu-bp.so.6.0.0 2023-01-24T14:47:22.101 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69efa000-7f6e69f04000 r-xp 00000000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:47:22.102 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e69f04000-7f6e6a103000 ---p 0000a000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:47:22.102 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a103000-7f6e6a104000 r--p 00009000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:47:22.102 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a104000-7f6e6a105000 rw-p 0000a000 08:01 98620 /usr/lib64/liblttng-ust-tracepoint.so.0.0.0 2023-01-24T14:47:22.102 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a105000-7f6e6a116000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.103 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a116000-7f6e6a11a000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.103 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a11a000-7f6e6a11c000 rw-s 00000000 00:0e 14549 anon_inode:[perf_event] 2023-01-24T14:47:22.103 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a11c000-7f6e6a28f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.103 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a296000-7f6e6a2d6000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.103 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a2db000-7f6e6a2fb000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.104 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a309000-7f6e6a319000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.104 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a31b000-7f6e6a33b000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.104 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a341000-7f6e6a391000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.104 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a395000-7f6e6a3a5000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.104 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e6a3a8000-7f6e73d35000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.105 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e73d35000-7f6e74515000 ---p 00000000 00:00 0 2023-01-24T14:47:22.105 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74515000-7f6e74867000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.105 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74867000-7f6e748ea000 r-xp 00000000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:47:22.105 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e748ea000-7f6e74ae9000 ---p 00083000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:47:22.105 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74ae9000-7f6e74aea000 r--p 00082000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:47:22.105 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74aea000-7f6e74aeb000 rw-p 00083000 08:01 6430 /usr/lib64/libpcre2-8.so.0.7.1 2023-01-24T14:47:22.106 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74aeb000-7f6e74af3000 r-xp 00000000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:47:22.106 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74af3000-7f6e74cf2000 ---p 00008000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:47:22.106 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74cf2000-7f6e74cf3000 r--p 00007000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:47:22.106 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74cf3000-7f6e74cf4000 rw-p 00008000 08:01 6983 /usr/lib64/libffi.so.6.0.2 2023-01-24T14:47:22.106 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74cf4000-7f6e74d1b000 r-xp 00000000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:47:22.107 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74d1b000-7f6e74f1b000 ---p 00027000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:47:22.107 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74f1b000-7f6e74f1c000 r--p 00027000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:47:22.107 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74f1c000-7f6e74f1d000 rw-p 00028000 08:01 6433 /usr/lib64/libselinux.so.1 2023-01-24T14:47:22.107 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74f1d000-7f6e74f1f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.107 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74f1f000-7f6e74fb5000 r-xp 00000000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:47:22.107 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e74fb5000-7f6e751b4000 ---p 00096000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:47:22.108 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e751b4000-7f6e751b6000 r--p 00095000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:47:22.108 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e751b6000-7f6e751b7000 rw-p 00097000 08:01 6901 /usr/lib64/libgmp.so.10.3.2 2023-01-24T14:47:22.108 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e751b7000-7f6e751e6000 r-xp 00000000 08:01 7093 /usr/lib64/libhogweed.so.4.5 2023-01-24T14:47:22.108 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e751e6000-7f6e753e5000 ---p 0002f000 08:01 7093 /usr/lib64/libhogweed.so.4.5 2023-01-24T14:47:22.108 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e753e5000-7f6e753e6000 r--p 0002e000 08:01 7093 /usr/lib64/libhogweed.so.4.5 2023-01-24T14:47:22.109 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e753e6000-7f6e753e7000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.109 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e753e7000-7f6e7541e000 r-xp 00000000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:47:22.109 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7541e000-7f6e7561e000 ---p 00037000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:47:22.109 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7561e000-7f6e75620000 r--p 00037000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:47:22.109 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75620000-7f6e75621000 rw-p 00039000 08:01 7094 /usr/lib64/libnettle.so.6.5 2023-01-24T14:47:22.109 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75621000-7f6e75633000 r-xp 00000000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:47:22.110 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75633000-7f6e75832000 ---p 00012000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:47:22.110 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75832000-7f6e75833000 r--p 00011000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:47:22.110 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75833000-7f6e75834000 rw-p 00012000 08:01 7122 /usr/lib64/libtasn1.so.6.5.5 2023-01-24T14:47:22.110 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75834000-7f6e759b1000 r-xp 00000000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:47:22.110 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e759b1000-7f6e75bb0000 ---p 0017d000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:47:22.111 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75bb0000-7f6e75bb4000 r--p 0017c000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:47:22.111 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75bb4000-7f6e75bb5000 rw-p 00180000 08:01 6922 /usr/lib64/libunistring.so.2.1.0 2023-01-24T14:47:22.111 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75bb5000-7f6e75bd1000 r-xp 00000000 08:01 6924 /usr/lib64/libidn2.so.0.3.6 2023-01-24T14:47:22.111 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75bd1000-7f6e75dd1000 ---p 0001c000 08:01 6924 /usr/lib64/libidn2.so.0.3.6 2023-01-24T14:47:22.111 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75dd1000-7f6e75dd2000 r--p 0001c000 08:01 6924 /usr/lib64/libidn2.so.0.3.6 2023-01-24T14:47:22.112 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75dd2000-7f6e75dd3000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.112 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75dd3000-7f6e75eea000 r-xp 00000000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:47:22.112 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e75eea000-7f6e760e9000 ---p 00117000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:47:22.112 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e760e9000-7f6e760f3000 r--p 00116000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:47:22.112 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e760f3000-7f6e760fd000 rw-p 00120000 08:01 6986 /usr/lib64/libp11-kit.so.0.3.0 2023-01-24T14:47:22.112 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e760fd000-7f6e76153000 r-xp 00000000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:47:22.113 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76153000-7f6e76352000 ---p 00056000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:47:22.113 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76352000-7f6e76355000 r--p 00055000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:47:22.113 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76355000-7f6e76356000 rw-p 00058000 08:01 1695 /usr/lib64/libmount.so.1.1.0 2023-01-24T14:47:22.113 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76356000-7f6e76357000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.113 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76357000-7f6e7635d000 r-xp 00000000 08:01 533 /usr/lib64/libuuid.so.1.3.0 2023-01-24T14:47:22.114 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7635d000-7f6e7655d000 ---p 00006000 08:01 533 /usr/lib64/libuuid.so.1.3.0 2023-01-24T14:47:22.114 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7655d000-7f6e7655e000 r--p 00006000 08:01 533 /usr/lib64/libuuid.so.1.3.0 2023-01-24T14:47:22.114 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7655e000-7f6e7655f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.114 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7655f000-7f6e7671b000 r-xp 00000000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:47:22.114 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7671b000-7f6e7691b000 ---p 001bc000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:47:22.114 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7691b000-7f6e7691f000 r--p 001bc000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:47:22.115 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7691f000-7f6e76921000 rw-p 001c0000 08:01 6477 /usr/lib64/libc-2.28.so 2023-01-24T14:47:22.115 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76921000-7f6e76925000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.115 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76925000-7f6e7693c000 r-xp 00000000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:47:22.115 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7693c000-7f6e76b3b000 ---p 00017000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:47:22.115 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76b3b000-7f6e76b3c000 r--p 00016000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:47:22.116 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76b3c000-7f6e76b3d000 rw-p 00017000 08:01 522 /usr/lib64/libgcc_s-8-20210514.so.1 2023-01-24T14:47:22.116 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76b3d000-7f6e76b89000 r-xp 00000000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:47:22.116 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76b89000-7f6e76d88000 ---p 0004c000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:47:22.116 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76d88000-7f6e76d8a000 r--p 0004b000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:47:22.116 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76d8a000-7f6e76d8d000 rw-p 0004d000 08:01 98584 /usr/lib64/libubsan.so.1.0.0 2023-01-24T14:47:22.117 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e76d8d000-7f6e7785f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.117 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7785f000-7f6e779e0000 r-xp 00000000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:47:22.117 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e779e0000-7f6e77bdf000 ---p 00181000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:47:22.117 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77bdf000-7f6e77be0000 r--p 00180000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:47:22.117 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77be0000-7f6e77be1000 rw-p 00181000 08:01 6479 /usr/lib64/libm-2.28.so 2023-01-24T14:47:22.117 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77be1000-7f6e77d67000 r-xp 00000000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:47:22.118 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77d67000-7f6e77f66000 ---p 00186000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:47:22.118 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77f66000-7f6e77f72000 r--p 00185000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:47:22.118 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77f72000-7f6e77f73000 rw-p 00191000 08:01 6656 /usr/lib64/libstdc++.so.6.0.25 2023-01-24T14:47:22.118 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77f73000-7f6e77f76000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.118 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77f76000-7f6e77f91000 r-xp 00000000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:47:22.119 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e77f91000-7f6e78190000 ---p 0001b000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:47:22.119 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78190000-7f6e78191000 r--p 0001a000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:47:22.119 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78191000-7f6e78192000 rw-p 0001b000 08:01 6487 /usr/lib64/libpthread-2.28.so 2023-01-24T14:47:22.119 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78192000-7f6e78196000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.119 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78196000-7f6e781a1000 r-xp 00000000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:47:22.119 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e781a1000-7f6e783a0000 ---p 0000b000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:47:22.120 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e783a0000-7f6e783a1000 r--p 0000a000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:47:22.120 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e783a1000-7f6e783a2000 rw-p 0000b000 08:01 7398 /usr/lib64/libnuma.so.1.0.0 2023-01-24T14:47:22.120 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e783a2000-7f6e78416000 r-xp 00000000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:47:22.120 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78416000-7f6e78616000 ---p 00074000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:47:22.121 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78616000-7f6e78618000 r--p 00074000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:47:22.121 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78618000-7f6e78619000 rw-p 00076000 08:01 98562 /usr/lib64/libyaml-cpp.so.0.6.2 2023-01-24T14:47:22.121 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78619000-7f6e7861a000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.121 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7861a000-7f6e78621000 r-xp 00000000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:47:22.121 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78621000-7f6e78820000 ---p 00007000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:47:22.122 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78820000-7f6e78821000 r--p 00006000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:47:22.122 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78821000-7f6e78822000 rw-p 00007000 08:01 6489 /usr/lib64/librt-2.28.so 2023-01-24T14:47:22.122 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78822000-7f6e78825000 r-xp 00000000 08:01 13879 /usr/lib64/libsctp.so.1.0.18 2023-01-24T14:47:22.122 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78825000-7f6e78a24000 ---p 00003000 08:01 13879 /usr/lib64/libsctp.so.1.0.18 2023-01-24T14:47:22.122 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78a24000-7f6e78a25000 r--p 00002000 08:01 13879 /usr/lib64/libsctp.so.1.0.18 2023-01-24T14:47:22.123 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78a25000-7f6e78a26000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.123 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78a26000-7f6e78a2c000 r-xp 00000000 08:01 98588 /usr/lib64/libatomic.so.1.2.0 2023-01-24T14:47:22.123 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78a2c000-7f6e78c2b000 ---p 00006000 08:01 98588 /usr/lib64/libatomic.so.1.2.0 2023-01-24T14:47:22.123 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78c2b000-7f6e78c2c000 r--p 00005000 08:01 98588 /usr/lib64/libatomic.so.1.2.0 2023-01-24T14:47:22.123 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78c2c000-7f6e78c2e000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.124 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78c2e000-7f6e78e0b000 r-xp 00000000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:47:22.124 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e78e0b000-7f6e7900a000 ---p 001dd000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:47:22.124 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7900a000-7f6e7901b000 r--p 001dc000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:47:22.124 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7901b000-7f6e7901d000 rw-p 001ed000 08:01 9274 /usr/lib64/libgnutls.so.30.28.2 2023-01-24T14:47:22.124 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7901d000-7f6e7901f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.125 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7901f000-7f6e79022000 r-xp 00000000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:47:22.125 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79022000-7f6e79221000 ---p 00003000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:47:22.125 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79221000-7f6e79222000 r--p 00002000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:47:22.125 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79222000-7f6e79223000 rw-p 00003000 08:01 6478 /usr/lib64/libdl-2.28.so 2023-01-24T14:47:22.126 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79223000-7f6e796e0000 r-xp 00000000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:47:22.126 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e796e0000-7f6e798df000 ---p 004bd000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:47:22.126 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e798df000-7f6e7992b000 r--p 004bc000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:47:22.126 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7992b000-7f6e7992c000 rw-p 00508000 08:01 98568 /usr/lib64/libcryptopp.so.8.6.0 2023-01-24T14:47:22.126 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7992c000-7f6e79930000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.127 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79930000-7f6e79942000 r-xp 00000000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:47:22.127 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79942000-7f6e79b41000 ---p 00012000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:47:22.127 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79b41000-7f6e79b42000 r--p 00011000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:47:22.127 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79b42000-7f6e79b43000 rw-p 00012000 08:01 7857 /usr/lib64/libcares.so.2.2.0 2023-01-24T14:47:22.127 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79b43000-7f6e79df7000 r-xp 00000000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:47:22.128 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79df7000-7f6e79ff7000 ---p 002b4000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:47:22.128 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e79ff7000-7f6e7a023000 r--p 002b4000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:47:22.128 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a023000-7f6e7a027000 rw-p 002e0000 08:01 1811 /usr/lib64/libcrypto.so.1.1.1k 2023-01-24T14:47:22.128 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a027000-7f6e7a02c000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.128 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a02c000-7f6e7a033000 r-xp 00000000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:47:22.129 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a033000-7f6e7a232000 ---p 00007000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:47:22.129 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a232000-7f6e7a233000 r--p 00006000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:47:22.129 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a233000-7f6e7a234000 rw-p 00007000 08:01 6650 /usr/lib64/libcap.so.2.48 2023-01-24T14:47:22.129 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a234000-7f6e7a236000 r-xp 00000000 08:01 7079 /usr/lib64/libaio.so.1.0.1 2023-01-24T14:47:22.129 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a236000-7f6e7a435000 ---p 00002000 08:01 7079 /usr/lib64/libaio.so.1.0.1 2023-01-24T14:47:22.130 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a435000-7f6e7a436000 r--p 00001000 08:01 7079 /usr/lib64/libaio.so.1.0.1 2023-01-24T14:47:22.130 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a436000-7f6e7a437000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.130 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a437000-7f6e7a4c9000 r-xp 00000000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:47:22.130 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a4c9000-7f6e7a6c9000 ---p 00092000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:47:22.130 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a6c9000-7f6e7a6cd000 r--p 00092000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:47:22.130 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a6cd000-7f6e7a6ce000 rw-p 00096000 08:01 13593 /usr/lib64/libudev.so.1.6.11 2023-01-24T14:47:22.131 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a6ce000-7f6e7a71b000 r-xp 00000000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:47:22.131 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a71b000-7f6e7a91a000 ---p 0004d000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:47:22.131 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a91a000-7f6e7a91f000 r--p 0004c000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:47:22.131 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a91f000-7f6e7a920000 rw-p 00051000 08:01 538 /usr/lib64/libblkid.so.1.1.0 2023-01-24T14:47:22.131 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a920000-7f6e7a921000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.132 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a921000-7f6e7a937000 r-xp 00000000 08:01 6611 /usr/lib64/libz.so.1.2.11 2023-01-24T14:47:22.132 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7a937000-7f6e7ab37000 ---p 00016000 08:01 6611 /usr/lib64/libz.so.1.2.11 2023-01-24T14:47:22.132 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ab37000-7f6e7ab38000 r--p 00016000 08:01 6611 /usr/lib64/libz.so.1.2.11 2023-01-24T14:47:22.132 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ab38000-7f6e7ab39000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.133 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ab39000-7f6e7ab55000 r-xp 00000000 08:01 7082 /usr/lib64/liblz4.so.1.8.3 2023-01-24T14:47:22.133 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ab55000-7f6e7ad54000 ---p 0001c000 08:01 7082 /usr/lib64/liblz4.so.1.8.3 2023-01-24T14:47:22.133 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ad54000-7f6e7ad55000 r--p 0001b000 08:01 7082 /usr/lib64/liblz4.so.1.8.3 2023-01-24T14:47:22.133 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ad55000-7f6e7ad56000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.133 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ad56000-7f6e7ad5f000 r-xp 00000000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:47:22.133 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7ad5f000-7f6e7af5f000 ---p 00009000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:47:22.134 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7af5f000-7f6e7af60000 r--p 00009000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:47:22.134 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7af60000-7f6e7af61000 rw-p 0000a000 08:01 7636 /usr/lib64/libsnappy.so.1.1.8 2023-01-24T14:47:22.134 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7af61000-7f6e7af85000 r--p 00000000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:47:22.134 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7af85000-7f6e7b05c000 r-xp 00024000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:47:22.134 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7b05c000-7f6e7b08e000 r--p 000fb000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:47:22.135 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7b08e000-7f6e7b08f000 ---p 0012d000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:47:22.135 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7b08f000-7f6e7b093000 r--p 0012d000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:47:22.135 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7b093000-7f6e7b096000 rw-p 00131000 08:01 98898 /usr/lib64/libasan.so.6.0.0 2023-01-24T14:47:22.135 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7b096000-7f6e7b94a000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.135 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7b94a000-7f6e7b977000 r-xp 00000000 08:01 6471 /usr/lib64/ld-2.28.so 2023-01-24T14:47:22.136 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7b977000-7f6e7bb6f000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.136 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7bb6f000-7f6e7bb77000 rw-p 00000000 00:00 0 2023-01-24T14:47:22.136 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7bb77000-7f6e7bb78000 r--p 0002d000 08:01 6471 /usr/lib64/ld-2.28.so 2023-01-24T14:47:22.136 INFO:tasks.ceph.osd.2.smithi105.stderr:7f6e7bb78000-7f6e7bb7a000 rw-p 0002e000 08:01 6471 /usr/lib64/ld-2.28.so 2023-01-24T14:47:22.136 INFO:tasks.ceph.osd.2.smithi105.stderr:7fffd3043000-7fffd30f4000 rw-p 00000000 00:00 0 [stack] 2023-01-24T14:47:22.137 INFO:tasks.ceph.osd.2.smithi105.stderr:7fffd31a4000-7fffd31a8000 r--p 00000000 00:00 0 [vvar] 2023-01-24T14:47:22.137 INFO:tasks.ceph.osd.2.smithi105.stderr:7fffd31a8000-7fffd31aa000 r-xp 00000000 00:00 0 [vdso] 2023-01-24T14:47:22.137 INFO:tasks.ceph.osd.2.smithi105.stderr:ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] 2023-01-24T14:47:22.235 INFO:tasks.ceph.osd.2.smithi105.stderr:daemon-helper: command crashed with signal 6 2023-01-24T14:47:24.100 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~140s 2023-01-24T14:47:24.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-01-24T14:47:29.403 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~146s 2023-01-24T14:47:29.404 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2023-01-24T14:47:34.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~151s 2023-01-24T14:47:34.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~11s 2023-01-24T14:47:40.008 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~156s 2023-01-24T14:47:40.009 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~16s 2023-01-24T14:47:45.311 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~162s 2023-01-24T14:47:45.312 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~21s 2023-01-24T14:47:50.614 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~167s 2023-01-24T14:47:50.615 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~27s 2023-01-24T14:47:55.917 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~172s 2023-01-24T14:47:55.918 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~32s 2023-01-24T14:48:01.220 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~178s 2023-01-24T14:48:01.221 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~37s 2023-01-24T14:48:06.523 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~183s 2023-01-24T14:48:06.524 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~42s 2023-01-24T14:48:11.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~188s 2023-01-24T14:48:11.826 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~48s 2023-01-24T14:48:17.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~194s 2023-01-24T14:48:17.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~53s 2023-01-24T14:48:17.753 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:17.752+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:22.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~199s 2023-01-24T14:48:22.432 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~58s 2023-01-24T14:48:22.754 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:22.752+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:27.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~204s 2023-01-24T14:48:27.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~64s 2023-01-24T14:48:27.755 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:27.753+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:32.756 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:32.754+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:33.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~209s 2023-01-24T14:48:33.037 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~69s 2023-01-24T14:48:37.758 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:37.756+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:38.339 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~215s 2023-01-24T14:48:38.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~74s 2023-01-24T14:48:42.759 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:42.757+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:43.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~220s 2023-01-24T14:48:43.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~80s 2023-01-24T14:48:47.760 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:47.758+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:48.945 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~225s 2023-01-24T14:48:48.946 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~85s 2023-01-24T14:48:52.761 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:52.759+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:54.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~231s 2023-01-24T14:48:54.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~90s 2023-01-24T14:48:57.763 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:48:57.761+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:48:59.550 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~236s 2023-01-24T14:48:59.550 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~95s 2023-01-24T14:49:02.764 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:02.763+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:04.853 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~241s 2023-01-24T14:49:04.853 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~101s 2023-01-24T14:49:07.765 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:07.764+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:10.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~247s 2023-01-24T14:49:10.155 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~106s 2023-01-24T14:49:12.766 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:12.764+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:15.458 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~252s 2023-01-24T14:49:15.458 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~111s 2023-01-24T14:49:17.768 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:17.766+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:20.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~257s 2023-01-24T14:49:20.761 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~117s 2023-01-24T14:49:22.769 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:22.767+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:26.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~262s 2023-01-24T14:49:26.064 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~122s 2023-01-24T14:49:27.770 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:27.768+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:31.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~268s 2023-01-24T14:49:31.366 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~127s 2023-01-24T14:49:32.771 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:32.769+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:36.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~273s 2023-01-24T14:49:36.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~133s 2023-01-24T14:49:37.772 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:37.770+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:41.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~278s 2023-01-24T14:49:41.972 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~138s 2023-01-24T14:49:42.773 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:42.771+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:47.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~284s 2023-01-24T14:49:47.275 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~143s 2023-01-24T14:49:47.774 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:47.773+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:52.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~289s 2023-01-24T14:49:52.579 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~148s 2023-01-24T14:49:52.776 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:52.774+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:57.777 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:49:57.776+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:49:57.881 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~294s 2023-01-24T14:49:57.882 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~154s 2023-01-24T14:50:02.778 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:50:02.776+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:50:03.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~300s 2023-01-24T14:50:03.185 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~159s 2023-01-24T14:50:07.779 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:50:07.777+0000 7f5314f63700 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.2 v2:172.21.15.105:6803/103822 for 29sec e78 v78) 2023-01-24T14:50:08.487 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~305s 2023-01-24T14:50:08.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~164s 2023-01-24T14:50:08.488 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2023-01-24T14:50:08.789 INFO:tasks.ceph.osd.0:Sent signal 15 2023-01-24T14:50:08.790 INFO:tasks.ceph.mon.a:Sent signal 15 2023-01-24T14:50:08.790 INFO:tasks.ceph.mgr.x:Sent signal 15 2023-01-24T14:50:08.791 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:50:08.789+0000 7f5317f69700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 103405) UID: 0 2023-01-24T14:50:08.791 INFO:tasks.ceph.mon.a.smithi105.stderr:2023-01-24T14:50:08.789+0000 7f5317f69700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-01-24T14:50:08.791 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:50:08.789+0000 7fb80cc0a700 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 103409) UID: 0 2023-01-24T14:50:08.792 INFO:tasks.ceph.mgr.x.smithi105.stderr:2023-01-24T14:50:08.789+0000 7fb80cc0a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-01-24T14:50:14.265 INFO:tasks.ceph.osd.0.smithi105.stderr:INFO 2023-01-24 14:50:14,264 [shard 0] osd - crimson shutdown complete 2023-01-24T14:50:15.048 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.048 INFO:tasks.ceph.osd.0.smithi105.stderr:================================================================= 2023-01-24T14:50:15.049 INFO:tasks.ceph.osd.0.smithi105.stderr:==103820==ERROR: LeakSanitizer: detected memory leaks 2023-01-24T14:50:15.049 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.049 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 1024 byte(s) in 2 object(s) allocated from: 2023-01-24T14:50:15.049 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.049 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7f51b1f in std::_Deque_base, std::allocator > >::_M_create_nodes(std::unique_ptr**, std::unique_ptr**) (/usr/bin/ceph-osd+0x3d099b1f) 2023-01-24T14:50:15.050 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.050 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 792 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.050 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.050 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7f391ee in crimson::net::SocketConnection::SocketConnection(crimson::net::SocketMessenger&, crimson::net::ChainedDispatchers&) (/usr/bin/ceph-osd+0x3d0811ee) 2023-01-24T14:50:15.050 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.050 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 736 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.051 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.051 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f2963b53 in ceph::common::intrusive_lru > >::get_or_create(hobject_t const&) (/usr/bin/ceph-osd+0x37aabb53) 2023-01-24T14:50:15.051 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.051 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 656 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.051 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.052 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f2debbc6 in crimson::osd::get_osd_priv(crimson::net::Connection*) (/usr/bin/ceph-osd+0x37f33bc6) 2023-01-24T14:50:15.052 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.052 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 512 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.052 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.052 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f4b5207f in CryptoAES::get_key_handler(ceph::buffer::v15_2_0::ptr const&, std::__cxx11::basic_string, std::allocator >&) (/usr/bin/ceph-osd+0x39c9a07f) 2023-01-24T14:50:15.053 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0xfffffffff () 2023-01-24T14:50:15.053 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.053 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 392 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.053 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.053 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7c39128 in crimson::net::FixedCPUServerSocket::accept const&)::{lambda(std::unique_ptr >, entity_addr_t)#1}>(crimson::net::SocketMessenger::start(boost::container::small_vector const&)::{lambda(std::unique_ptr >, entity_addr_t)#1}&&)::{lambda(auto:1&)#1}::operator()(crimson::net::SocketMessenger::start(boost::container::small_vector const&)::{lambda(std::unique_ptr >, entity_addr_t)#1}&)::{lambda()#1}::operator()()::{lambda()#1}::operator()()::{lambda(seastar::accept_result)#1}::operator()(seastar::accept_result)::{lambda()#1}::operator()() (/usr/bin/ceph-osd+0x3cd81128) 2023-01-24T14:50:15.054 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.054 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 344 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.054 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.054 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f2140aa1 in PGBackend::load_metadata(hobject_t const&)::{lambda()#2}::operator()() const (/usr/bin/ceph-osd+0x37288aa1) 2023-01-24T14:50:15.054 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.055 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 264 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.055 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.055 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f2d798d0 in seastar::shared_ptr seastar::shared_ptr_make_helper::make&, watch_info_t&, entity_name_t const&, boost::intrusive_ptr >(crimson::osd::Watch::private_ctag_t&&, boost::intrusive_ptr&, watch_info_t&, entity_name_t const&, boost::intrusive_ptr&&) (/usr/bin/ceph-osd+0x37ec18d0) 2023-01-24T14:50:15.060 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x7ffe78d845df ([stack]+0xab5df) 2023-01-24T14:50:15.060 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.060 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 256 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.060 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.060 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7c4cbb5 in crimson::net::FixedCPUServerSocket::accept const&)::{lambda(std::unique_ptr >, entity_addr_t)#1}>(crimson::net::SocketMessenger::start(boost::container::small_vector const&)::{lambda(std::unique_ptr >, entity_addr_t)#1}&&)::{lambda(auto:1&)#1}::operator()(crimson::net::SocketMessenger::start(boost::container::small_vector const&)::{lambda(std::unique_ptr >, entity_addr_t)#1}&)::{lambda()#1}::operator()()::{lambda()#1}::operator()()::{lambda(seastar::accept_result)#1}::operator()(seastar::accept_result) (/usr/bin/ceph-osd+0x3cd94bb5) 2023-01-24T14:50:15.061 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.061 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 160 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.061 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.061 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f80d955d in crimson::net::ProtocolV2::execute_accepting()::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const (/usr/bin/ceph-osd+0x3d22155d) 2023-01-24T14:50:15.061 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x7ffe78d83f2f ([stack]+0xaaf2f) 2023-01-24T14:50:15.062 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.062 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 120 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.062 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.062 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x562700177143 in seastar::reactor_backend_epoll::make_pollable_fd_state(seastar::file_desc, seastar::pollable_fd_state::speculation) (/usr/bin/ceph-osd+0x452bf143) 2023-01-24T14:50:15.062 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x562700176fdb in seastar::reactor_backend_aio::make_pollable_fd_state(seastar::file_desc, seastar::pollable_fd_state::speculation) (/usr/bin/ceph-osd+0x452befdb) 2023-01-24T14:50:15.063 INFO:tasks.ceph.osd.0.smithi105.stderr: #3 0x7ffe78d8593f ([stack]+0xac93f) 2023-01-24T14:50:15.063 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.063 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 104 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.063 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319e69c in __interceptor_posix_memalign (/lib64/libasan.so.6+0xb569c) 2023-01-24T14:50:15.063 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f4b81657 in ceph::buffer::v15_2_0::create_aligned_in_mempool(unsigned int, unsigned int, int) (/usr/bin/ceph-osd+0x39cc9657) 2023-01-24T14:50:15.064 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x5626f4b824b4 in ceph::buffer::v15_2_0::create_aligned(unsigned int, unsigned int) (/usr/bin/ceph-osd+0x39cca4b4) 2023-01-24T14:50:15.064 INFO:tasks.ceph.osd.0.smithi105.stderr: #3 0x6030004ab05f () 2023-01-24T14:50:15.064 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.064 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 104 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.064 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.065 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f2d192ed in std::_Rb_tree_iterator const, watch_info_t> > std::_Rb_tree, std::pair const, watch_info_t>, std::_Select1st const, watch_info_t> >, std::less >, std::allocator const, watch_info_t> > >::_M_emplace_hint_unique const&>, std::tuple >(std::_Rb_tree_const_iterator const, watch_info_t> >, std::piecewise_construct_t const&, std::tuple const&>&&, std::tuple&&) (/usr/bin/ceph-osd+0x37e612ed) 2023-01-24T14:50:15.065 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x56270074597f (/usr/bin/ceph-osd+0x4588d97f) 2023-01-24T14:50:15.065 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.065 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 96 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.065 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.066 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7f73f9d in crimson::net::Protocol::close(bool, std::optional >) (/usr/bin/ceph-osd+0x3d0bbf9d) 2023-01-24T14:50:15.066 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.066 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 96 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.066 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.066 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f1546a25 in seastar::shared_promise<>::shared_promise() (/usr/bin/ceph-osd+0x3668ea25) 2023-01-24T14:50:15.067 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x7ffe78d84a17 ([stack]+0xaba17) 2023-01-24T14:50:15.067 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.067 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 96 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.067 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.067 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f82053dc in seastar::lw_shared_ptr::shared_state> seastar::lw_shared_ptr::shared_state>::make >(seastar::future&&) (/usr/bin/ceph-osd+0x3d34d3dc) 2023-01-24T14:50:15.068 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x7ffe78d84baf ([stack]+0xabbaf) 2023-01-24T14:50:15.068 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.068 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 72 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.068 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.068 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f2d12f29 in std::_Rb_tree_iterator const, seastar::shared_ptr > > std::_Rb_tree, std::pair const, seastar::shared_ptr >, std::_Select1st const, seastar::shared_ptr > >, std::less >, std::allocator const, seastar::shared_ptr > > >::_M_emplace_hint_unique const&>, std::tuple >(std::_Rb_tree_const_iterator const, seastar::shared_ptr > >, std::piecewise_construct_t const&, std::tuple const&>&&, std::tuple&&) (/usr/bin/ceph-osd+0x37e5af29) 2023-01-24T14:50:15.068 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x6080020dd18f () 2023-01-24T14:50:15.069 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.069 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 64 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.069 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.069 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7f51dc0 in std::_Deque_base, std::allocator > >::_M_initialize_map(unsigned long) (/usr/bin/ceph-osd+0x3d099dc0) 2023-01-24T14:50:15.069 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x61400015063f () 2023-01-24T14:50:15.070 INFO:tasks.ceph.osd.0.smithi105.stderr: #3 0x5626f7f3af43 in crimson::net::SocketConnection::SocketConnection(crimson::net::SocketMessenger&, crimson::net::ChainedDispatchers&) (/usr/bin/ceph-osd+0x3d082f43) 2023-01-24T14:50:15.070 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.070 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 64 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.070 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.070 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7f51dc0 in std::_Deque_base, std::allocator > >::_M_initialize_map(unsigned long) (/usr/bin/ceph-osd+0x3d099dc0) 2023-01-24T14:50:15.073 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x61400015063f () 2023-01-24T14:50:15.074 INFO:tasks.ceph.osd.0.smithi105.stderr: #3 0x5626f7f3a0f7 in crimson::net::SocketConnection::SocketConnection(crimson::net::SocketMessenger&, crimson::net::ChainedDispatchers&) (/usr/bin/ceph-osd+0x3d0820f7) 2023-01-24T14:50:15.074 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.074 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 56 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.074 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.074 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626ffae28cc in seastar::net::posix_server_socket_impl::accept()::{lambda(std::tuple)#1}::operator()(std::tuple) const (/usr/bin/ceph-osd+0x44c2a8cc) 2023-01-24T14:50:15.075 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x690f15ac6edf0001 () 2023-01-24T14:50:15.075 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.075 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 48 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.075 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.075 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626ffaebd7c in seastar::net::posix_connected_socket_impl::source(seastar::connected_socket_input_stream_config) (/usr/bin/ceph-osd+0x44c33d7c) 2023-01-24T14:50:15.076 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0x7fd5c31ac7ec in __asan::GetCurrentThread() (/lib64/libasan.so.6+0xc37ec) 2023-01-24T14:50:15.076 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.076 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 40 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.076 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.076 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626ffb0a7ac in std::_MakeUniq::__single_object std::make_unique*&>(int&, seastar::socket_address&, seastar::pollable_fd&&, seastar::server_socket::load_balancing_algorithm&, unsigned int&, std::pmr::polymorphic_allocator*&) (/usr/bin/ceph-osd+0x44c527ac) 2023-01-24T14:50:15.077 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.077 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 24 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.077 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.077 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626ffaefb14 in seastar::net::posix_connected_socket_impl::sink() (/usr/bin/ceph-osd+0x44c37b14) 2023-01-24T14:50:15.077 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.078 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 24 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.078 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.078 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f4b4a0ee in CryptoKey::_set_secret(int, ceph::buffer::v15_2_0::ptr const&) (/usr/bin/ceph-osd+0x39c920ee) 2023-01-24T14:50:15.078 INFO:tasks.ceph.osd.0.smithi105.stderr: #2 0xfffffffff () 2023-01-24T14:50:15.078 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.078 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 16 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.079 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.079 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626f7ef14a7 in cephx_verify_authorizer(crimson::common::CephContext*, KeyStore const&, ceph::buffer::v15_2_0::list::iterator_impl&, unsigned long, CephXServiceTicketInfo&, std::unique_ptr >*, std::__cxx11::basic_string, std::allocator >*, ceph::buffer::v15_2_0::list*) (/usr/bin/ceph-osd+0x3d0394a7) 2023-01-24T14:50:15.079 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.079 INFO:tasks.ceph.osd.0.smithi105.stderr:Indirect leak of 4 byte(s) in 1 object(s) allocated from: 2023-01-24T14:50:15.079 INFO:tasks.ceph.osd.0.smithi105.stderr: #0 0x7fd5c319f307 in operator new(unsigned long) (/lib64/libasan.so.6+0xb6307) 2023-01-24T14:50:15.080 INFO:tasks.ceph.osd.0.smithi105.stderr: #1 0x5626ffb0ae48 in std::_MakeUniq::__single_object std::make_unique*&>(int&, seastar::socket_address&, seastar::pollable_fd&&, seastar::server_socket::load_balancing_algorithm&, unsigned int&, std::pmr::polymorphic_allocator*&) (/usr/bin/ceph-osd+0x44c52e48) 2023-01-24T14:50:15.080 INFO:tasks.ceph.osd.0.smithi105.stderr: 2023-01-24T14:50:15.080 INFO:tasks.ceph.osd.0.smithi105.stderr:SUMMARY: AddressSanitizer: 6164 byte(s) leaked in 27 allocation(s). 2023-01-24T14:50:15.199 INFO:tasks.ceph.osd.0.smithi105.stderr:daemon-helper: command failed with exit status 1 2023-01-24T17:43:18.150 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-01-24T17:43:18.170 INFO:tasks.workunit:Stopping ['rbd/test_librbd.sh'] on client.0... 2023-01-24T17:43:18.170 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-01-24T17:43:18.434 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 103, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 82, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/test_librbd.sh) on smithi105 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=510284b66513490445619d1430aa869868c71a09 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' 2023-01-24T17:43:18.502 ERROR:teuthology.run_tasks: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=48a675fd43574ebd8e00ae9ef0b4b584 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 103, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 82, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/test_librbd.sh) on smithi105 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=510284b66513490445619d1430aa869868c71a09 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh' 2023-01-24T17:43:18.505 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-01-24T17:43:18.511 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-01-24T17:43:18.511 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T17:45:18.549 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-01-24T17:45:18.550 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1902, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2302, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2536, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1609, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1600, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2023-01-24T17:45:18.590 INFO:teuthology.misc:Shutting down mds daemons... 2023-01-24T17:45:18.591 INFO:teuthology.misc:Shutting down osd daemons... 2023-01-24T17:45:18.591 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-01-24T17:45:18.592 INFO:teuthology.orchestra.run:waiting for 300 2023-01-24T17:45:18.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T17:45:18.594 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1441, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1902, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2302, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2536, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1609, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1600, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0' 2023-01-24T17:45:18.603 INFO:tasks.ceph.osd.0:Stopped 2023-01-24T17:45:18.604 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-01-24T17:45:18.604 INFO:teuthology.orchestra.run:waiting for 300 2023-01-24T17:45:18.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T17:45:18.605 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1441, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1902, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2302, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2536, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1609, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1600, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1' 2023-01-24T17:45:18.613 INFO:tasks.ceph.osd.1:Stopped 2023-01-24T17:45:18.614 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-01-24T17:45:18.614 INFO:teuthology.orchestra.run:waiting for 300 2023-01-24T17:45:18.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T17:45:18.615 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1441, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1902, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2302, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2536, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1609, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1600, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2' 2023-01-24T17:45:18.624 INFO:tasks.ceph.osd.2:Stopped 2023-01-24T17:45:18.624 INFO:teuthology.misc:Shutting down mgr daemons... 2023-01-24T17:45:18.625 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-01-24T17:45:18.625 INFO:teuthology.orchestra.run:waiting for 300 2023-01-24T17:45:18.626 INFO:tasks.ceph.mgr.x:Stopped 2023-01-24T17:45:18.626 INFO:teuthology.misc:Shutting down mon daemons... 2023-01-24T17:45:18.627 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-01-24T17:45:18.627 INFO:teuthology.orchestra.run:waiting for 300 2023-01-24T17:45:18.628 INFO:tasks.ceph.mon.a:Stopped 2023-01-24T17:45:18.628 INFO:tasks.ceph:Checking cluster log for badness... 2023-01-24T17:45:18.629 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'is full \(reached quota' | egrep -v '\(POOL_FULL\)' | head -n 1 2023-01-24T17:45:18.667 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T17:45:18.668 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-01-24T17:45:18.855 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T17:45:18.856 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-01-24T17:45:18.945 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T17:45:18.945 DEBUG:teuthology.orchestra.run.smithi105:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-01-24T17:45:19.017 INFO:tasks.ceph:Archiving mon data... 2023-01-24T17:45:19.019 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311/data/mon.a.tgz 2023-01-24T17:45:19.019 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2023-01-24T17:45:19.041 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.dhDcrRGJai 2023-01-24T17:45:19.042 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.dhDcrRGJai 2023-01-24T17:45:19.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.dhDcrRGJai 2023-01-24T17:45:19.273 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.dhDcrRGJai is 297KB 2023-01-24T17:45:19.341 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.dhDcrRGJai 2023-01-24T17:45:19.359 INFO:tasks.ceph:Cleaning ceph cluster... 2023-01-24T17:45:19.359 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-01-24T17:45:19.425 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-01-24T17:45:19.426 DEBUG:teuthology.orchestra.run.smithi105:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-01-24T17:45:19.493 INFO:teuthology.orchestra.run.smithi105.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-01-24T17:45:19.499 INFO:tasks.ceph:Archiving crash dumps... 2023-01-24T17:45:19.501 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/crash to /home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311/remote/smithi105/crash 2023-01-24T17:45:19.503 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/lib/ceph/crash -- . 2023-01-24T17:45:19.568 INFO:tasks.ceph:Compressing logs... 2023-01-24T17:45:19.569 DEBUG:teuthology.orchestra.run.smithi105:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-01-24T17:45:22.403 INFO:tasks.ceph:Archiving logs... 2023-01-24T17:45:22.404 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311/remote/smithi105/log 2023-01-24T17:45:22.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /var/log/ceph -- . 2023-01-24T17:45:22.859 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1902, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2302, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2536, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1609, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1600, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2023-01-24T17:45:22.868 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-01-24T17:45:22.880 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/contextutil.py", line 33, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/task/install/__init__.py", line 619, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1902, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2302, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2536, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1609, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1600, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2023-01-24T17:45:22.884 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits... 2023-01-24T17:45:22.885 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits 2023-01-24T17:45:22.953 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-01-24T17:45:22.966 INFO:teuthology.task.clock:Checking final clock skew... 2023-01-24T17:45:22.967 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T17:45:22.985 INFO:teuthology.orchestra.run.smithi105.stderr:bash: ntpq: command not found 2023-01-24T17:45:22.990 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T17:45:22.990 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2023-01-24T17:45:22.991 INFO:teuthology.orchestra.run.smithi105.stdout:^- hv01.front.sepia.ceph.com 3 6 377 17 +4765us[+4740us] +/- 92ms 2023-01-24T17:45:22.991 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 7 377 10 -244us[ -268us] +/- 62ms 2023-01-24T17:45:22.991 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +195us[ +171us] +/- 47ms 2023-01-24T17:45:22.991 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T17:45:22.993 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-01-24T17:45:23.002 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-01-24T17:45:23.002 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-01-24T17:45:23.011 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-01-24T17:45:23.117 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T17:45:23.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T17:45:23.150 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-01-24T17:45:23.161 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-01-24T17:45:23.171 INFO:teuthology.task.internal:Duration was 12363.408528 seconds 2023-01-24T17:45:23.172 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-01-24T17:45:23.180 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-01-24T17:45:23.181 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-01-24T17:45:23.237 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T17:45:23.534 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-01-24T17:45:23.535 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2023-01-24T17:45:23.535 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-01-24T17:45:23.558 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-01-24T17:45:23.559 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-01-24T17:45:23.641 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-01-24T17:45:23.641 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-01-24T17:45:23.751 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-01-24T17:45:23.766 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-01-24T17:45:23.767 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-01-24T17:45:23.795 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-01-24T17:45:23.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-01-24T17:45:23.862 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2023-01-24T17:45:23.875 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2023-01-24T17:45:23.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T17:45:23.933 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-01-24T17:45:23.944 INFO:teuthology.task.internal:Transferring archived files... 2023-01-24T17:45:23.945 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311/remote/smithi105 2023-01-24T17:45:23.946 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar cz -f - -C /home/ubuntu/cephtest/archive -- . 2023-01-24T17:45:24.025 INFO:teuthology.task.internal:Removing archive directory... 2023-01-24T17:45:24.026 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2023-01-24T17:45:24.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-01-24T17:45:24.092 INFO:teuthology.task.internal:Not uploading archives. 2023-01-24T17:45:24.093 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-01-24T17:45:24.104 INFO:teuthology.task.internal:Tidying up after the test... 2023-01-24T17:45:24.105 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-01-24T17:45:24.138 INFO:teuthology.orchestra.run.smithi105.stdout: 525248 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jan 24 17:45 /home/ubuntu/cephtest 2023-01-24T17:45:24.138 INFO:teuthology.orchestra.run.smithi105.stdout: 788241 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 24 14:42 /home/ubuntu/cephtest/mnt.0 2023-01-24T17:45:24.139 INFO:teuthology.orchestra.run.smithi105.stdout: 788242 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Jan 24 14:43 /home/ubuntu/cephtest/mnt.0/client.0 2023-01-24T17:45:24.139 INFO:teuthology.orchestra.run.smithi105.stdout: 788284 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Jan 24 14:43 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T17:45:24.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T17:45:24.141 INFO:teuthology.orchestra.run.smithi105.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-01-24T17:45:24.141 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/task/internal/__init__.py", line 49, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph.py", line 1902, in task ctx.managers[config['cluster']].stop_pg_num_changes() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2302, in stop_pg_num_changes osd_dump = self.get_osd_dump_json() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 2536, in get_osd_dump_json out = self.raw_cluster_cmd('osd', 'dump', '--format=json') File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1609, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/git.ceph.com_ceph-c_510284b66513490445619d1430aa869868c71a09/qa/tasks/ceph_manager.py", line 1600, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/remote.py", line 525, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/run_tasks.py", line 188, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/task/internal/__init__.py", line 54, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8e1bba12e584b7ae912e92e94afe9c075ab51fa3/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi105 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-01-24T17:45:24.155 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-01-24T17:45:24.167 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi105 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json' 2023-01-24T17:45:24.167 INFO:teuthology.nuke:Checking targets against current locks 2023-01-24T17:45:24.200 DEBUG:teuthology.nuke:shortname: smithi105 2023-01-24T17:45:24.201 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T17:45:24.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rzarzynski-2023-01-24_14:08:39-crimson-rados-main-distro-default-smithi/7135311', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 14:11:25.142536', 'locked_by': 'scheduled_rzarzynski@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCjmRpV2nHf7zNb3iCW6xG4he1DmKPZ5qBdD8qr1YwZ1S7uwiCJ5kV1hlufB/jJbinbEvAiq3NmQjHfUZpWVJvEvQUguaoqr2pag64PI5VS3FSCB4nFiCXD5zqYASyxVG1EcUSBqJBi8Hh3DQoumOVbT6J1di3pqgo13JsATpZLBd342M1EoYyW1ia/0D5llnKznvJq996rtGm2RXZ50u+1VxpteD3Pf/tZuIe2hgJThto6ZZsoCBtNf0MuIWY7hqJpltje6lhzMCkmK9zYkccrYrRtDj2LORcXaWrv6Hw6a0sR8fAHQ/v7Tb9lyTH6wFAcCRom2xN9GuWhqyVJkJRaudordVVaj2tTKFZIQ2/ax4K59QnSK9KCyS/+nBg+tVxQTdBYIKA5rNii8xymJZ8iB+nBlpmBVgTA/ilanqp95QtsMYb/Na68E0nnm/1o5nkb9E3/oQPtN8xPGmVvmomyvtN667o9Nnzas3jR0CV/L1OGwmF0Ba96de4Bti0Qu8k='} 2023-01-24T17:45:24.289 INFO:teuthology.orchestra.console:Power off smithi105 2023-01-24T17:45:24.290 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-01-24T17:45:24.311 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-01-24T17:45:28.334 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-01-24T17:45:32.448 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-01-24T17:45:36.564 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-01-24T17:45:36.774 INFO:teuthology.orchestra.console:Power off for smithi105 completed 2023-01-24T17:45:36.880 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{centos_latest clusters/fixed-1 crimson_qa_overrides deploy/ceph tasks/rbd_api_tests_old_format} duration: 12363.408528327942 failure_reason: 'Command failed (workunit test rbd/test_librbd.sh) on smithi105 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=510284b66513490445619d1430aa869868c71a09 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd.sh''' flavor: crimson owner: scheduled_rzarzynski@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=48a675fd43574ebd8e00ae9ef0b4b584 status: fail success: false 2023-01-24T17:45:36.881 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T17:45:36.937 INFO:teuthology.run:FAIL