2023-01-25T11:25:17.478 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T11:25:17.479 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/e2e} --owner scheduled_yuriw@teuthology --verbose --name yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137348 -- /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137348/orig.config.yaml 2023-01-25T11:25:17.517 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T11:25:17.590 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137348 branch: wip-yuri6-testing-2023-01-24-0746 description: rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/e2e} email: yweinstein@radhat.com first_in_suite: false job_id: '7137348' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri6-testing-2023-01-24-0746 ceph: conf: client: debug client: 20 debug mgrc: 20 debug ms: 1 global: mon election default strategy: 3 mds: debug mgrc: 20 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug mgrc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2023-01-24-0746 sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - host.a - osd.0 - osd.1 - osd.2 - mon.a - mgr.a - client.0 - - host.b - client.1 sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri6-testing-2023-01-24-0746 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c targets: smithi119.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4G0QD3mRAbkRPeA8lvzPwCDb0oVV9HjHViHmUACndbzUaSD9ZY2nTjzQT5IGERjm68EQ4nkMKnpscJ7bvvYz35bW6f9C9nlPppDw3j0q1lgbV+gi4z7VMWPAO9H8hfy7SC7oE/s1KSl7p5sJIRTXzDY6bHNPTUWUOW9M9ER7q3W4eaTpPNfinoaj4RwcG3yLIxXnUh29WGdJtqoI+lfHbj/LBsWP4Y2YVbtHVs07QPru80KleuwBFtfd3rwuW0ImZVznn3dgSufk3kuXWBAC8RZ7lYf8kGLo5r8Mn+Bo+NyIr5Brl2ojccTuUZFYM51swxz7WcxQZOND/kPQo0ThVIOuRthwPpctlzcA2T0L0OiilJGgH6rS/W4bz99cc4cGs0ruaMyTmDdyuyRlUmXRCuiRAd11kan9tzvmPkPdhBAcw9Na8fdQhUuMSeGclZocPBzWKHkKuELm8IDpeArzb3HRTeNnpcdULuUVnST/TjSB+oAPF5e29SIG6Urzncl8= smithi157.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtNzImittQlLcj4uKtYSS38dIACmEly5ErdJxOAI1nCFQ2nQCY0aMJVfkDXUhEDx+xiIV54Avb5tDLmW+29ET8FIvM9pfN6Gs3+IMTaSQE5ihEH1YN1T7iwBegrOokfRBipqWl7ITgDjYuHC81PLFOem4hPU27c2hKznkeVzG0OUVveZCsXAXjmO/jx7SzSN+P2L5iiFFY+bng5pyxCdMvQMkarLJYdhsSgdAolVUgs59KmfujaDj7yLw4CEPz5kgiPWJMuLo9UI6ROp+Pu2jum0RVXphh4NnZK9CuxD7CXZAbwt2Jrvj0KW8mSE7PaXt6gZAfcvzpCC3r1NOVsIxaKw5NJ0SqunN/IJKBCmZpVa7LbWJWfchZiqXwn8BAkdpthpKdbkRH97w+8Ceg4pScj+NSrTOxkvrpJu4qYZ9j39H77SLaZHbQC7i4IUcA+w7qmWQO6nEkDUH07gc0Pml7pbVtEhGbTv5Q37bmj0mLHtTyWTcSnx+cdYAH6x9ZA40= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - sudo sed -i 's/runtime = "runc"/#runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/#runtime = "crun"/runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: null - workunit: clients: client.1: - cephadm/create_iscsi_disks.sh - workunit: clients: client.0: - cephadm/test_dashboard_e2e.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:27:30 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T11:25:17.590 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa; will attempt to use it 2023-01-25T11:25:17.591 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_44e4095da12a8269eb6cfe163cbb0d00b1a6412c/qa/tasks 2023-01-25T11:25:17.591 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T11:25:17.593 INFO:teuthology.task.internal:Checking packages... 2023-01-25T11:25:17.621 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '44e4095da12a8269eb6cfe163cbb0d00b1a6412c' 2023-01-25T11:25:17.621 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T11:25:17.622 INFO:teuthology.packaging:ref: None 2023-01-25T11:25:17.622 INFO:teuthology.packaging:tag: None 2023-01-25T11:25:17.622 INFO:teuthology.packaging:branch: wip-yuri6-testing-2023-01-24-0746 2023-01-25T11:25:17.622 INFO:teuthology.packaging:sha1: 44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:25:17.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri6-testing-2023-01-24-0746 2023-01-25T11:25:17.775 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1986.g44e4095d 2023-01-25T11:25:17.775 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T11:25:17.782 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T11:25:17.782 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T11:25:17.790 INFO:teuthology.task.internal:Saving configuration 2023-01-25T11:25:17.804 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T11:25:17.810 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T11:25:17.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 11:19:35.955725', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIeQKOVFlpuZTJwdZIjz0m1u4RZRrZFospK92UXSJbGYQeECoUOmIDTxRuf45hC/+0Ok9ykXPNZKq7QnUpkN8qg='} 2023-01-25T11:25:17.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:27:30-rados-wip-yuri6-testing-2023-01-24-0746-distro-default-smithi/7137348', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 11:19:35.957319', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD45Zt5HS2RJ7ckFTaX8c3IRcUAHtXhCTc9LI20cWQUDOvpfsFCrGhuCwWSbHUz06t7dd8XfkJnfODa0JyoeL9g='} 2023-01-25T11:25:17.854 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T11:25:17.861 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['host.a', 'osd.0', 'osd.1', 'osd.2', 'mon.a', 'mgr.a', 'client.0'] 2023-01-25T11:25:17.861 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['host.b', 'client.1'] 2023-01-25T11:25:17.861 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T11:25:17.926 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f79d76f5700>, signals=[15]) 2023-01-25T11:25:17.927 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T11:25:17.933 INFO:teuthology.task.internal:Opening connections... 2023-01-25T11:25:17.934 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2023-01-25T11:25:17.935 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:25:18.001 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2023-01-25T11:25:18.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:25:18.069 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T11:25:18.076 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-01-25T11:25:18.096 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-01-25T11:25:18.097 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-01-25T11:25:18.155 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2023-01-25T11:25:18.155 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8" 2023-01-25T11:25:18.155 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2023-01-25T11:25:18.156 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2023-01-25T11:25:18.156 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8" 2023-01-25T11:25:18.156 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2023-01-25T11:25:18.156 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T11:25:18.156 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-01-25T11:25:18.157 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T11:25:18.157 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2023-01-25T11:25:18.157 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T11:25:18.157 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T11:25:18.157 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T11:25:18.158 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-01-25T11:25:18.173 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2023-01-25T11:25:18.194 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2023-01-25T11:25:18.194 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2023-01-25T11:25:18.254 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="CentOS Stream" 2023-01-25T11:25:18.254 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="8" 2023-01-25T11:25:18.254 INFO:teuthology.orchestra.run.smithi157.stdout:ID="centos" 2023-01-25T11:25:18.255 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE="rhel fedora" 2023-01-25T11:25:18.255 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="8" 2023-01-25T11:25:18.255 INFO:teuthology.orchestra.run.smithi157.stdout:PLATFORM_ID="platform:el8" 2023-01-25T11:25:18.255 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="CentOS Stream 8" 2023-01-25T11:25:18.255 INFO:teuthology.orchestra.run.smithi157.stdout:ANSI_COLOR="0;31" 2023-01-25T11:25:18.255 INFO:teuthology.orchestra.run.smithi157.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-01-25T11:25:18.256 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://centos.org/" 2023-01-25T11:25:18.256 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T11:25:18.256 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T11:25:18.256 INFO:teuthology.orchestra.run.smithi157.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-01-25T11:25:18.257 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2023-01-25T11:25:18.271 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T11:25:18.280 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T11:25:18.286 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T11:25:18.286 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2023-01-25T11:25:18.288 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2023-01-25T11:25:18.314 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T11:25:18.325 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T11:25:18.325 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2023-01-25T11:25:18.347 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2023-01-25T11:25:18.385 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T11:25:18.435 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T11:25:18.449 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T11:25:18.450 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T11:25:18.450 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T11:25:18.451 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T11:25:18.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T11:25:18.471 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-01-25T11:25:18.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T11:25:18.472 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-01-25T11:25:18.530 INFO:teuthology.orchestra.run.smithi119.stdout:4.18.0-408.el8.x86_64 2023-01-25T11:25:18.531 INFO:teuthology.task.kernel:Running kernel on smithi119: 4.18.0-408.el8.x86_64 2023-01-25T11:25:18.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum install -y kernel 2023-01-25T11:25:18.532 INFO:teuthology.orchestra.run.smithi157.stdout:4.18.0-408.el8.x86_64 2023-01-25T11:25:18.532 INFO:teuthology.task.kernel:Running kernel on smithi157: 4.18.0-408.el8.x86_64 2023-01-25T11:25:18.533 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum install -y kernel 2023-01-25T11:25:22.172 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - AppStream 35 MB/s | 27 MB 00:00 2023-01-25T11:25:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - AppStream 31 MB/s | 27 MB 00:00 2023-01-25T11:25:27.888 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - BaseOS 33 MB/s | 27 MB 00:00 2023-01-25T11:25:28.023 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - BaseOS 32 MB/s | 27 MB 00:00 2023-01-25T11:25:32.195 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - Extras 161 kB/s | 18 kB 00:00 2023-01-25T11:25:32.297 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - Extras 159 kB/s | 18 kB 00:00 2023-01-25T11:25:32.581 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - PowerTools 22 MB/s | 5.6 MB 00:00 2023-01-25T11:25:32.708 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - PowerTools 20 MB/s | 5.6 MB 00:00 2023-01-25T11:25:33.797 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-01-25T11:25:33.923 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-01-25T11:25:34.830 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 15 MB/s | 13 MB 00:00 2023-01-25T11:25:34.983 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 14 MB/s | 13 MB 00:00 2023-01-25T11:25:38.122 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 428 kB/s | 24 kB 00:00 2023-01-25T11:25:38.333 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 316 kB/s | 24 kB 00:00 2023-01-25T11:25:41.270 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T11:25:41.271 INFO:teuthology.orchestra.run.smithi119.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T11:25:41.432 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:25:41.432 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:25:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:25:41.433 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:25:41.433 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:25:41.433 INFO:teuthology.orchestra.run.smithi119.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T11:25:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T11:25:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T11:25:41.434 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:25:41.434 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:25:41.434 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:25:41.435 INFO:teuthology.orchestra.run.smithi119.stdout:Install 3 Packages 2023-01-25T11:25:41.435 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:25:41.436 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 85 M 2023-01-25T11:25:41.436 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 94 M 2023-01-25T11:25:41.436 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:25:41.489 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2023-01-25T11:25:41.490 INFO:teuthology.orchestra.run.smithi157.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2023-01-25T11:25:41.644 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:25:41.645 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:25:41.645 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:25:41.645 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:25:41.645 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:25:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: kernel x86_64 4.18.0-448.el8 CentOS-BaseOS 9.2 M 2023-01-25T11:25:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-core x86_64 4.18.0-448.el8 CentOS-BaseOS 41 M 2023-01-25T11:25:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules x86_64 4.18.0-448.el8 CentOS-BaseOS 34 M 2023-01-25T11:25:41.646 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:25:41.647 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:25:41.647 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:25:41.647 INFO:teuthology.orchestra.run.smithi157.stdout:Install 3 Packages 2023-01-25T11:25:41.647 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:25:41.648 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 85 M 2023-01-25T11:25:41.648 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 94 M 2023-01-25T11:25:41.648 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:25:41.815 INFO:teuthology.orchestra.run.smithi119.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 24 MB/s | 9.2 MB 00:00 2023-01-25T11:25:42.576 INFO:teuthology.orchestra.run.smithi157.stdout:(1/3): kernel-4.18.0-448.el8.x86_64.rpm 9.9 MB/s | 9.2 MB 00:00 2023-01-25T11:25:42.915 INFO:teuthology.orchestra.run.smithi119.stdout:(2/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 23 MB/s | 34 MB 00:01 2023-01-25T11:25:43.317 INFO:teuthology.orchestra.run.smithi157.stdout:(2/3): kernel-core-4.18.0-448.el8.x86_64.rpm 25 MB/s | 41 MB 00:01 2023-01-25T11:25:43.324 INFO:teuthology.orchestra.run.smithi119.stdout:(3/3): kernel-core-4.18.0-448.el8.x86_64.rpm 22 MB/s | 41 MB 00:01 2023-01-25T11:25:43.327 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:25:43.327 INFO:teuthology.orchestra.run.smithi119.stdout:Total 45 MB/s | 85 MB 00:01 2023-01-25T11:25:43.723 INFO:teuthology.orchestra.run.smithi157.stdout:(3/3): kernel-modules-4.18.0-448.el8.x86_64.rpm 16 MB/s | 34 MB 00:02 2023-01-25T11:25:43.725 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:25:43.725 INFO:teuthology.orchestra.run.smithi157.stdout:Total 41 MB/s | 85 MB 00:02 2023-01-25T11:25:43.962 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:25:44.356 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:25:44.356 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:25:44.359 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:25:44.759 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:25:44.759 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:25:46.437 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:25:46.474 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:25:46.844 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:25:46.869 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:25:49.203 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:25:49.643 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:25:49.804 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T11:25:50.314 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T11:25:53.622 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T11:25:54.061 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T11:25:54.145 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 1/3 2023-01-25T11:25:54.572 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T11:25:57.383 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T11:25:57.644 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:25:57.898 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 2/3 2023-01-25T11:25:58.180 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:19.304 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:19.330 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:19.738 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T11:26:19.739 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T11:26:20.203 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-core-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:20.229 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:20.335 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:20.335 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:26:20.335 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:26:20.336 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T11:26:20.336 INFO:teuthology.orchestra.run.smithi119.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T11:26:20.336 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:26:20.336 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:26:20.451 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel --last | head -n 1 2023-01-25T11:26:20.690 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 11:25:57 AM UTC 2023-01-25T11:26:20.691 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T11:26:20.692 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T11:26:20.692 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T11:26:20.692 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T11:26:20.693 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2023-01-25T11:26:20.693 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T11:26:20.693 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q kernel --last | head -n 1 2023-01-25T11:26:20.881 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 11:25:57 AM UTC 2023-01-25T11:26:20.882 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T11:26:20.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2023-01-25T11:26:20.915 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2023-01-25T11:26:20.916 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2023-01-25T11:26:20.916 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.02 2023-01-25T11:26:20.916 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 129.el8 2023-01-25T11:26:20.917 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2023-01-25T11:26:20.917 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T11:26:20.917 INFO:teuthology.orchestra.run.smithi119.stdout:Group : System Environment/Base 2023-01-25T11:26:20.917 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 9265586 2023-01-25T11:26:20.917 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2023-01-25T11:26:20.918 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T11:26:20.918 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T11:26:20.918 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T11:26:20.918 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T11:26:20.918 INFO:teuthology.orchestra.run.smithi119.stdout:Relocations : (not relocatable) 2023-01-25T11:26:20.919 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : CentOS BuildSystem 2023-01-25T11:26:20.919 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2023-01-25T11:26:20.919 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T11:26:20.919 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2023-01-25T11:26:20.919 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2023-01-25T11:26:20.920 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:26:20.920 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T11:26:20.920 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T11:26:20.920 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T11:26:20.920 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2023-01-25T11:26:20.921 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:26:20.921 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T11:26:20.922 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T11:26:20.922 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 4.18.0-448.el8.x86_64 2023-01-25T11:26:20.922 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T11:26:20.960 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: kernel-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:20.960 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-4.18.0-448.el8.x86_64 1/3 2023-01-25T11:26:20.960 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-core-4.18.0-448.el8.x86_64 2/3 2023-01-25T11:26:21.534 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : kernel-modules-4.18.0-448.el8.x86_64 3/3 2023-01-25T11:26:21.535 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:26:21.535 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:26:21.535 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-4.18.0-448.el8.x86_64 kernel-core-4.18.0-448.el8.x86_64 2023-01-25T11:26:21.535 INFO:teuthology.orchestra.run.smithi157.stdout: kernel-modules-4.18.0-448.el8.x86_64 2023-01-25T11:26:21.536 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:26:21.536 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:26:21.662 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel --last | head -n 1 2023-01-25T11:26:21.859 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 11:25:57 AM UTC 2023-01-25T11:26:21.860 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T11:26:21.860 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-408.el8.x86_64 Expected: 4.18.0-448.el8.x86_64 2023-01-25T11:26:21.861 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-01-25T11:26:21.861 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-01-25T11:26:21.861 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2023-01-25T11:26:21.861 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi157.front.sepia.ceph.com, path=None, version=distro) 2023-01-25T11:26:21.862 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q kernel --last | head -n 1 2023-01-25T11:26:22.049 INFO:teuthology.orchestra.run.smithi157.stdout:kernel-4.18.0-448.el8.x86_64 Wed 25 Jan 2023 11:25:57 AM UTC 2023-01-25T11:26:22.051 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-448.el8.x86_64 2023-01-25T11:26:22.051 DEBUG:teuthology.orchestra.run.smithi157:> sudo rpm -qi grub2-tools 2023-01-25T11:26:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:Name : grub2-tools 2023-01-25T11:26:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:Epoch : 1 2023-01-25T11:26:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:Version : 2.02 2023-01-25T11:26:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:Release : 129.el8 2023-01-25T11:26:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:Architecture: x86_64 2023-01-25T11:26:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:Install Date: Sun 28 Aug 2022 01:03:50 PM UTC 2023-01-25T11:26:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:Group : System Environment/Base 2023-01-25T11:26:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:Size : 9265586 2023-01-25T11:26:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:License : GPLv3+ 2023-01-25T11:26:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-01-25T11:26:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-01-25T11:26:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-01-25T11:26:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:Build Host : kbuilder.bsys.centos.org 2023-01-25T11:26:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:Relocations : (not relocatable) 2023-01-25T11:26:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:Packager : CentOS BuildSystem 2023-01-25T11:26:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:Vendor : CentOS 2023-01-25T11:26:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:URL : http://www.gnu.org/software/grub/ 2023-01-25T11:26:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:Summary : Support tools for GRUB. 2023-01-25T11:26:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:Description : 2023-01-25T11:26:22.089 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:26:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-01-25T11:26:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:customizable bootloader with modular architecture. It supports a rich 2023-01-25T11:26:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:variety of kernel formats, file systems, computer architectures and 2023-01-25T11:26:22.090 INFO:teuthology.orchestra.run.smithi157.stdout:hardware devices. 2023-01-25T11:26:22.090 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:26:22.090 INFO:teuthology.orchestra.run.smithi157.stdout:This subpackage provides tools for support of all platforms. 2023-01-25T11:26:22.091 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-01-25T11:26:22.091 INFO:teuthology.task.kernel:Updating grub on smithi157 to boot 4.18.0-448.el8.x86_64 2023-01-25T11:26:22.091 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-01-25T11:26:22.714 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2023-01-25T11:26:23.464 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2023-01-25T11:26:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:done 2023-01-25T11:26:24.944 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-01-25T11:26:24.967 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.bzklUY6pEh 2023-01-25T11:26:24.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bzklUY6pEh 2023-01-25T11:26:25.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.bzklUY6pEh 2023-01-25T11:26:25.192 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.bzklUY6pEh is 6KB 2023-01-25T11:26:25.239 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.bzklUY6pEh 2023-01-25T11:26:25.256 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T11:26:25.285 INFO:teuthology.orchestra.run.smithi157.stderr:done 2023-01-25T11:26:25.287 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2023-01-25T11:26:25.310 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.BseQAiPMGT 2023-01-25T11:26:25.310 DEBUG:teuthology.orchestra.run.smithi157:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.BseQAiPMGT 2023-01-25T11:26:25.337 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T11:26:25.338 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T11:26:25.338 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T11:26:25.338 INFO:teuthology.orchestra.run.smithi119.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T11:26:25.339 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T11:26:25.394 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.BseQAiPMGT 2023-01-25T11:26:25.499 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.BseQAiPMGT is 6KB 2023-01-25T11:26:25.512 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2023-01-25T11:26:25.546 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.BseQAiPMGT 2023-01-25T11:26:25.563 DEBUG:teuthology.orchestra.run.smithi157:> sudo /bin/ls /boot/loader/entries || true 2023-01-25T11:26:25.628 INFO:teuthology.orchestra.run.smithi157.stdout:2ff282740cf84d4fa02ff60ed95fbbff-0-rescue.conf 2023-01-25T11:26:25.628 INFO:teuthology.orchestra.run.smithi157.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-305.3.1.el8.x86_64.conf 2023-01-25T11:26:25.629 INFO:teuthology.orchestra.run.smithi157.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-408.el8.x86_64.conf 2023-01-25T11:26:25.629 INFO:teuthology.orchestra.run.smithi157.stdout:2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64.conf 2023-01-25T11:26:25.630 DEBUG:teuthology.orchestra.run.smithi157:> sudo grub2-set-default 2ff282740cf84d4fa02ff60ed95fbbff-4.18.0-448.el8.x86_64 2023-01-25T11:26:25.791 DEBUG:teuthology.orchestra.run.smithi157:> sudo shutdown -r now 2023-01-25T11:26:55.544 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2023-01-25T11:26:55.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-01-25T11:26:55.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:26:55.794 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2023-01-25T11:26:55.795 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-01-25T11:26:55.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:27:30.408 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-01-25T11:27:55.572 DEBUG:teuthology.orchestra.remote:timed out 2023-01-25T11:28:00.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-01-25T11:28:00.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:28:18.792 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2023-01-25T11:28:25.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-01-25T11:28:25.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:28:25.876 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-01-25T11:28:26.157 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-01-25T11:28:26.158 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-448.el8.x86_64"... 2023-01-25T11:28:26.158 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-01-25T11:28:26.177 INFO:teuthology.orchestra.run.smithi119.stdout:4.18.0-448.el8.x86_64 2023-01-25T11:28:26.177 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T11:28:26.177 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T11:28:26.178 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T11:28:27.178 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2023-01-25T11:28:27.179 DEBUG:teuthology.orchestra.run.smithi119:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T11:28:27.270 INFO:teuthology.orchestra.run.smithi119.stdout:ttyS1 2023-01-25T11:28:27.284 DEBUG:teuthology.parallel:result is None 2023-01-25T11:28:48.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-01-25T11:28:48.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:28:49.111 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-01-25T11:28:49.368 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-01-25T11:28:49.369 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-448.el8.x86_64"... 2023-01-25T11:28:49.369 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2023-01-25T11:28:49.387 INFO:teuthology.orchestra.run.smithi157.stdout:4.18.0-448.el8.x86_64 2023-01-25T11:28:49.387 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-448.el8.x86_64 vs 4.18.0-448.el8.x86_64 2023-01-25T11:28:49.388 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-01-25T11:28:49.388 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-01-25T11:28:50.389 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2023-01-25T11:28:50.390 DEBUG:teuthology.orchestra.run.smithi157:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T11:28:50.491 INFO:teuthology.orchestra.run.smithi157.stdout:ttyS1 2023-01-25T11:28:50.503 DEBUG:teuthology.parallel:result is None 2023-01-25T11:28:50.503 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T11:28:50.510 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T11:28:50.512 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T11:28:50.514 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T11:28:50.534 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T11:28:50.541 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T11:28:50.550 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T11:28:50.551 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T11:28:50.573 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T11:28:50.606 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T11:28:50.612 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T11:28:50.613 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T11:28:50.634 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T11:28:50.683 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T11:28:50.684 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T11:28:50.693 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T11:28:50.695 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T11:28:50.696 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T11:28:50.708 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T11:28:50.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T11:28:50.711 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T11:28:50.766 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T11:28:50.775 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T11:28:50.776 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T11:28:50.779 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T11:28:50.825 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T11:28:50.877 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T11:28:50.933 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T11:28:51.013 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T11:28:51.060 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-01-25T11:28:51.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T11:28:51.127 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T11:28:51.172 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T11:28:51.233 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T11:28:51.280 DEBUG:teuthology.orchestra.run.smithi157:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T11:28:51.322 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-01-25T11:28:51.322 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T11:28:51.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2023-01-25T11:28:51.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2023-01-25T11:28:51.455 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T11:28:51.501 INFO:teuthology.orchestra.run.smithi157.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T11:28:51.961 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T11:28:51.969 INFO:teuthology.task.internal:Starting timer... 2023-01-25T11:28:51.969 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T11:28:51.996 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T11:28:52.004 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T11:28:52.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2023-01-25T11:28:52.085 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2023-01-25T11:28:52.086 DEBUG:teuthology.orchestra.run.smithi157:> sudo service auditd rotate 2023-01-25T11:28:52.171 INFO:teuthology.orchestra.run.smithi157.stdout:Rotating logs: 2023-01-25T11:28:52.172 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T11:28:52.173 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2023-01-25T11:28:52.208 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2023-01-25T11:28:52.209 DEBUG:teuthology.orchestra.run.smithi157:> /usr/sbin/getenforce 2023-01-25T11:28:52.238 INFO:teuthology.orchestra.run.smithi157.stdout:Permissive 2023-01-25T11:28:52.239 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi157.front.sepia.ceph.com': 'permissive'} 2023-01-25T11:28:52.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T11:28:52.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T11:28:52.277 DEBUG:teuthology.orchestra.run.smithi157:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T11:28:52.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T11:28:52.307 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T11:28:52.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2023-01-25T11:28:52.353 DEBUG:teuthology.orchestra.run.smithi157:> sudo /usr/sbin/setenforce permissive 2023-01-25T11:28:52.387 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T11:28:52.397 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-25T11:28:52.403 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-25T11:28:52.518 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T11:28:52.535 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T11:28:52.536 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T11:35:18.690 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2023-01-25T11:35:18.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-01-25T11:35:18.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:35:18.763 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-01-25T11:35:18.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2023-01-25T11:35:18.836 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T11:35:18.902 DEBUG:teuthology.orchestra.run.smithi157:> true 2023-01-25T11:35:18.974 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T11:35:18.985 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T11:35:18.986 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T11:35:18.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T11:35:18.989 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T11:35:18.990 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T11:35:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T11:35:19.034 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T11:35:19.047 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T11:35:19.062 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T11:35:19.104 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2023-01-25T11:35:19.116 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2023-01-25T11:35:19.124 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: ntpd: command not found 2023-01-25T11:35:19.129 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T11:35:19.136 INFO:teuthology.orchestra.run.smithi157.stdout:506 Cannot talk to daemon 2023-01-25T11:35:19.143 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T11:35:19.150 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T11:35:19.163 INFO:teuthology.orchestra.run.smithi157.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T11:35:19.205 INFO:teuthology.orchestra.run.smithi119.stderr:bash: ntpq: command not found 2023-01-25T11:35:19.209 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T11:35:19.209 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-01-25T11:35:19.209 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.210 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.210 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.210 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.237 INFO:teuthology.orchestra.run.smithi157.stderr:bash: ntpq: command not found 2023-01-25T11:35:19.241 INFO:teuthology.orchestra.run.smithi157.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T11:35:19.241 INFO:teuthology.orchestra.run.smithi157.stdout:=============================================================================== 2023-01-25T11:35:19.241 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.242 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.242 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.242 INFO:teuthology.orchestra.run.smithi157.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T11:35:19.243 INFO:teuthology.run_tasks:Running task pexec... 2023-01-25T11:35:19.254 INFO:teuthology.task.pexec:Executing custom commands... 2023-01-25T11:35:19.255 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi119.front.sepia.ceph.com 2023-01-25T11:35:19.255 DEBUG:teuthology.orchestra.run.smithi119:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-01-25T11:35:19.256 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi157.front.sepia.ceph.com 2023-01-25T11:35:19.256 DEBUG:teuthology.orchestra.run.smithi157:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-01-25T11:35:19.997 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:13 ago on Wed 25 Jan 2023 11:34:06 AM UTC. 2023-01-25T11:35:20.033 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:13 ago on Wed 25 Jan 2023 11:34:06 AM UTC. 2023-01-25T11:35:21.444 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:35:21.445 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:35:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2023-01-25T11:35:21.446 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:35:21.446 INFO:teuthology.orchestra.run.smithi119.stdout:Resetting modules: 2023-01-25T11:35:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: container-tools 2023-01-25T11:35:21.446 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:35:21.447 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:35:21.447 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:35:21.447 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:35:21.492 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:35:21.493 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:35:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2023-01-25T11:35:21.494 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:35:21.494 INFO:teuthology.orchestra.run.smithi157.stdout:Resetting modules: 2023-01-25T11:35:21.494 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools 2023-01-25T11:35:21.495 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:35:21.495 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:35:21.495 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:35:21.495 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:35:21.707 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:35:21.779 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:35:22.504 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:16 ago on Wed 25 Jan 2023 11:34:06 AM UTC. 2023-01-25T11:35:22.583 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:16 ago on Wed 25 Jan 2023 11:34:06 AM UTC. 2023-01-25T11:35:23.921 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:35:23.924 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================================= 2023-01-25T11:35:23.924 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:35:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================================= 2023-01-25T11:35:23.925 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-01-25T11:35:23.925 INFO:teuthology.orchestra.run.smithi119.stdout: conmon x86_64 3:2.1.4-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 57 k 2023-01-25T11:35:23.925 INFO:teuthology.orchestra.run.smithi119.stdout: container-selinux noarch 2:2.189.0-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 60 k 2023-01-25T11:35:23.925 INFO:teuthology.orchestra.run.smithi119.stdout: containernetworking-plugins x86_64 1:1.1.1-3.module_el8.7.0+1216+b022c01d CentOS-AppStream 18 M 2023-01-25T11:35:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: criu x86_64 3.15-3.module_el8.7.0+1216+b022c01d CentOS-AppStream 518 k 2023-01-25T11:35:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-overlayfs x86_64 1.9-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 73 k 2023-01-25T11:35:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: libslirp x86_64 4.4.0-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 70 k 2023-01-25T11:35:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: runc x86_64 1:1.1.4-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 2.9 M 2023-01-25T11:35:23.926 INFO:teuthology.orchestra.run.smithi119.stdout: slirp4netns x86_64 1.2.0-2.module_el8.7.0+1216+b022c01d CentOS-AppStream 54 k 2023-01-25T11:35:23.926 INFO:teuthology.orchestra.run.smithi119.stdout:Installing group/module packages: 2023-01-25T11:35:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: aardvark-dns x86_64 2:1.1.0-4.module_el8.7.0+1216+b022c01d CentOS-AppStream 997 k 2023-01-25T11:35:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: buildah x86_64 1:1.27.0-2.module_el8.7.0+1216+b022c01d CentOS-AppStream 8.1 M 2023-01-25T11:35:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: cockpit-podman noarch 53-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 547 k 2023-01-25T11:35:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: crun x86_64 1.5-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 213 k 2023-01-25T11:35:23.927 INFO:teuthology.orchestra.run.smithi119.stdout: netavark x86_64 2:1.1.0-6.module_el8.7.0+1216+b022c01d CentOS-AppStream 2.2 M 2023-01-25T11:35:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: python3-podman noarch 4.2.0-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 151 k 2023-01-25T11:35:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: skopeo x86_64 2:1.9.2-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 6.8 M 2023-01-25T11:35:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: toolbox x86_64 0.0.99.3-1.module_el8.5.0+877+1c30e0c9 CentOS-AppStream 2.3 M 2023-01-25T11:35:23.928 INFO:teuthology.orchestra.run.smithi119.stdout: udica noarch 0.2.6-3.module_el8.7.0+1216+b022c01d CentOS-AppStream 49 k 2023-01-25T11:35:23.928 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:35:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 CentOS-AppStream 25 k 2023-01-25T11:35:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyxdg noarch 0.25-16.el8 CentOS-AppStream 94 k 2023-01-25T11:35:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: yajl x86_64 2.1.0-11.el8 CentOS-AppStream 41 k 2023-01-25T11:35:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:Installing module profiles: 2023-01-25T11:35:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: container-tools/common 2023-01-25T11:35:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling module streams: 2023-01-25T11:35:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: container-tools rhel8 2023-01-25T11:35:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:35:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:35:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================================= 2023-01-25T11:35:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:Install 12 Packages 2023-01-25T11:35:23.931 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 8 Packages 2023-01-25T11:35:23.931 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:35:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 43 M 2023-01-25T11:35:23.932 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:35:23.990 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:35:23.993 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================================= 2023-01-25T11:35:23.993 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:35:23.993 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================================= 2023-01-25T11:35:23.994 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-01-25T11:35:23.994 INFO:teuthology.orchestra.run.smithi157.stdout: conmon x86_64 3:2.1.4-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 57 k 2023-01-25T11:35:23.994 INFO:teuthology.orchestra.run.smithi157.stdout: container-selinux noarch 2:2.189.0-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 60 k 2023-01-25T11:35:23.994 INFO:teuthology.orchestra.run.smithi157.stdout: containernetworking-plugins x86_64 1:1.1.1-3.module_el8.7.0+1216+b022c01d CentOS-AppStream 18 M 2023-01-25T11:35:23.994 INFO:teuthology.orchestra.run.smithi157.stdout: criu x86_64 3.15-3.module_el8.7.0+1216+b022c01d CentOS-AppStream 518 k 2023-01-25T11:35:23.995 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-overlayfs x86_64 1.9-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 73 k 2023-01-25T11:35:23.995 INFO:teuthology.orchestra.run.smithi157.stdout: libslirp x86_64 4.4.0-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 70 k 2023-01-25T11:35:23.995 INFO:teuthology.orchestra.run.smithi157.stdout: runc x86_64 1:1.1.4-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 2.9 M 2023-01-25T11:35:23.995 INFO:teuthology.orchestra.run.smithi157.stdout: slirp4netns x86_64 1.2.0-2.module_el8.7.0+1216+b022c01d CentOS-AppStream 54 k 2023-01-25T11:35:23.995 INFO:teuthology.orchestra.run.smithi157.stdout:Installing group/module packages: 2023-01-25T11:35:23.996 INFO:teuthology.orchestra.run.smithi157.stdout: aardvark-dns x86_64 2:1.1.0-4.module_el8.7.0+1216+b022c01d CentOS-AppStream 997 k 2023-01-25T11:35:23.996 INFO:teuthology.orchestra.run.smithi157.stdout: buildah x86_64 1:1.27.0-2.module_el8.7.0+1216+b022c01d CentOS-AppStream 8.1 M 2023-01-25T11:35:23.996 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman noarch 53-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 547 k 2023-01-25T11:35:23.996 INFO:teuthology.orchestra.run.smithi157.stdout: crun x86_64 1.5-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 213 k 2023-01-25T11:35:23.996 INFO:teuthology.orchestra.run.smithi157.stdout: netavark x86_64 2:1.1.0-6.module_el8.7.0+1216+b022c01d CentOS-AppStream 2.2 M 2023-01-25T11:35:23.997 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman noarch 4.2.0-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 151 k 2023-01-25T11:35:23.997 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo x86_64 2:1.9.2-1.module_el8.7.0+1216+b022c01d CentOS-AppStream 6.8 M 2023-01-25T11:35:23.997 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox x86_64 0.0.99.3-1.module_el8.5.0+877+1c30e0c9 CentOS-AppStream 2.3 M 2023-01-25T11:35:23.997 INFO:teuthology.orchestra.run.smithi157.stdout: udica noarch 0.2.6-3.module_el8.7.0+1216+b022c01d CentOS-AppStream 49 k 2023-01-25T11:35:23.997 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:35:23.998 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 CentOS-AppStream 25 k 2023-01-25T11:35:23.998 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg noarch 0.25-16.el8 CentOS-AppStream 94 k 2023-01-25T11:35:23.998 INFO:teuthology.orchestra.run.smithi157.stdout: yajl x86_64 2.1.0-11.el8 CentOS-AppStream 41 k 2023-01-25T11:35:23.998 INFO:teuthology.orchestra.run.smithi157.stdout:Installing module profiles: 2023-01-25T11:35:23.998 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools/common 2023-01-25T11:35:23.999 INFO:teuthology.orchestra.run.smithi157.stdout:Enabling module streams: 2023-01-25T11:35:23.999 INFO:teuthology.orchestra.run.smithi157.stdout: container-tools rhel8 2023-01-25T11:35:23.999 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:35:23.999 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:35:23.999 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================================= 2023-01-25T11:35:23.999 INFO:teuthology.orchestra.run.smithi157.stdout:Install 12 Packages 2023-01-25T11:35:24.000 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 8 Packages 2023-01-25T11:35:24.000 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:35:24.000 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 43 M 2023-01-25T11:35:24.001 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:35:24.043 INFO:teuthology.orchestra.run.smithi119.stdout:(1/20): aardvark-dns-1.1.0-4.module_el8.7.0+121 8.5 MB/s | 997 kB 00:00 2023-01-25T11:35:24.076 INFO:teuthology.orchestra.run.smithi119.stdout:(2/20): crun-1.5-1.module_el8.7.0+1216+b022c01d 6.3 MB/s | 213 kB 00:00 2023-01-25T11:35:24.099 INFO:teuthology.orchestra.run.smithi157.stdout:(1/20): aardvark-dns-1.1.0-4.module_el8.7.0+121 9.6 MB/s | 997 kB 00:00 2023-01-25T11:35:24.110 INFO:teuthology.orchestra.run.smithi119.stdout:(3/20): cockpit-podman-53-1.module_el8.7.0+1216 3.0 MB/s | 547 kB 00:00 2023-01-25T11:35:24.125 INFO:teuthology.orchestra.run.smithi157.stdout:(2/20): crun-1.5-1.module_el8.7.0+1216+b022c01d 8.3 MB/s | 213 kB 00:00 2023-01-25T11:35:24.143 INFO:teuthology.orchestra.run.smithi119.stdout:(4/20): python3-podman-4.2.0-1.module_el8.7.0+1 4.4 MB/s | 151 kB 00:00 2023-01-25T11:35:24.169 INFO:teuthology.orchestra.run.smithi119.stdout:(5/20): python3-pytoml-0.1.14-5.git7dea353.el8. 982 kB/s | 25 kB 00:00 2023-01-25T11:35:24.200 INFO:teuthology.orchestra.run.smithi157.stdout:(3/20): netavark-1.1.0-6.module_el8.7.0+1216+b0 30 MB/s | 2.2 MB 00:00 2023-01-25T11:35:24.202 INFO:teuthology.orchestra.run.smithi119.stdout:(6/20): python3-pyxdg-0.25-16.el8.noarch.rpm 2.8 MB/s | 94 kB 00:00 2023-01-25T11:35:24.217 INFO:teuthology.orchestra.run.smithi157.stdout:(4/20): python3-podman-4.2.0-1.module_el8.7.0+1 8.9 MB/s | 151 kB 00:00 2023-01-25T11:35:24.242 INFO:teuthology.orchestra.run.smithi157.stdout:(5/20): python3-pytoml-0.1.14-5.git7dea353.el8. 990 kB/s | 25 kB 00:00 2023-01-25T11:35:24.259 INFO:teuthology.orchestra.run.smithi157.stdout:(6/20): python3-pyxdg-0.25-16.el8.noarch.rpm 5.4 MB/s | 94 kB 00:00 2023-01-25T11:35:24.269 INFO:teuthology.orchestra.run.smithi119.stdout:(7/20): netavark-1.1.0-6.module_el8.7.0+1216+b0 11 MB/s | 2.2 MB 00:00 2023-01-25T11:35:24.376 INFO:teuthology.orchestra.run.smithi157.stdout:(7/20): cockpit-podman-53-1.module_el8.7.0+1216 1.4 MB/s | 547 kB 00:00 2023-01-25T11:35:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:(8/20): toolbox-0.0.99.3-1.module_el8.5.0+877+1 15 MB/s | 2.3 MB 00:00 2023-01-25T11:35:24.453 INFO:teuthology.orchestra.run.smithi119.stdout:(9/20): udica-0.2.6-3.module_el8.7.0+1216+b022c 1.4 MB/s | 49 kB 00:00 2023-01-25T11:35:24.487 INFO:teuthology.orchestra.run.smithi119.stdout:(10/20): yajl-2.1.0-11.el8.x86_64.rpm 1.2 MB/s | 41 kB 00:00 2023-01-25T11:35:24.521 INFO:teuthology.orchestra.run.smithi119.stdout:(11/20): conmon-2.1.4-1.module_el8.7.0+1216+b02 1.7 MB/s | 57 kB 00:00 2023-01-25T11:35:24.536 INFO:teuthology.orchestra.run.smithi157.stdout:(8/20): skopeo-1.9.2-1.module_el8.7.0+1216+b022 25 MB/s | 6.8 MB 00:00 2023-01-25T11:35:24.577 INFO:teuthology.orchestra.run.smithi157.stdout:(9/20): toolbox-0.0.99.3-1.module_el8.5.0+877+1 11 MB/s | 2.3 MB 00:00 2023-01-25T11:35:24.602 INFO:teuthology.orchestra.run.smithi157.stdout:(10/20): udica-0.2.6-3.module_el8.7.0+1216+b022 736 kB/s | 49 kB 00:00 2023-01-25T11:35:24.638 INFO:teuthology.orchestra.run.smithi119.stdout:(12/20): buildah-1.27.0-2.module_el8.7.0+1216+b 11 MB/s | 8.1 MB 00:00 2023-01-25T11:35:24.653 INFO:teuthology.orchestra.run.smithi157.stdout:(11/20): yajl-2.1.0-11.el8.x86_64.rpm 542 kB/s | 41 kB 00:00 2023-01-25T11:35:24.663 INFO:teuthology.orchestra.run.smithi119.stdout:(13/20): container-selinux-2.189.0-1.module_el8 425 kB/s | 60 kB 00:00 2023-01-25T11:35:24.678 INFO:teuthology.orchestra.run.smithi157.stdout:(12/20): conmon-2.1.4-1.module_el8.7.0+1216+b02 751 kB/s | 57 kB 00:00 2023-01-25T11:35:24.703 INFO:teuthology.orchestra.run.smithi157.stdout:(13/20): container-selinux-2.189.0-1.module_el8 1.2 MB/s | 60 kB 00:00 2023-01-25T11:35:24.705 INFO:teuthology.orchestra.run.smithi119.stdout:(14/20): criu-3.15-3.module_el8.7.0+1216+b022c0 12 MB/s | 518 kB 00:00 2023-01-25T11:35:24.738 INFO:teuthology.orchestra.run.smithi119.stdout:(15/20): fuse-overlayfs-1.9-1.module_el8.7.0+12 2.2 MB/s | 73 kB 00:00 2023-01-25T11:35:24.764 INFO:teuthology.orchestra.run.smithi119.stdout:(16/20): libslirp-4.4.0-1.module_el8.7.0+1216+b 2.7 MB/s | 70 kB 00:00 2023-01-25T11:35:24.803 INFO:teuthology.orchestra.run.smithi157.stdout:(14/20): buildah-1.27.0-2.module_el8.7.0+1216+b 10 MB/s | 8.1 MB 00:00 2023-01-25T11:35:24.837 INFO:teuthology.orchestra.run.smithi157.stdout:(15/20): fuse-overlayfs-1.9-1.module_el8.7.0+12 2.2 MB/s | 73 kB 00:00 2023-01-25T11:35:24.862 INFO:teuthology.orchestra.run.smithi157.stdout:(16/20): criu-3.15-3.module_el8.7.0+1216+b022c0 3.2 MB/s | 518 kB 00:00 2023-01-25T11:35:24.887 INFO:teuthology.orchestra.run.smithi157.stdout:(17/20): libslirp-4.4.0-1.module_el8.7.0+1216+b 1.4 MB/s | 70 kB 00:00 2023-01-25T11:35:24.921 INFO:teuthology.orchestra.run.smithi157.stdout:(18/20): slirp4netns-1.2.0-2.module_el8.7.0+121 1.6 MB/s | 54 kB 00:00 2023-01-25T11:35:24.939 INFO:teuthology.orchestra.run.smithi119.stdout:(17/20): skopeo-1.9.2-1.module_el8.7.0+1216+b02 9.2 MB/s | 6.8 MB 00:00 2023-01-25T11:35:24.973 INFO:teuthology.orchestra.run.smithi119.stdout:(18/20): slirp4netns-1.2.0-2.module_el8.7.0+121 1.6 MB/s | 54 kB 00:00 2023-01-25T11:35:25.065 INFO:teuthology.orchestra.run.smithi119.stdout:(19/20): runc-1.1.4-1.module_el8.7.0+1216+b022c 9.8 MB/s | 2.9 MB 00:00 2023-01-25T11:35:25.113 INFO:teuthology.orchestra.run.smithi157.stdout:(19/20): runc-1.1.4-1.module_el8.7.0+1216+b022c 12 MB/s | 2.9 MB 00:00 2023-01-25T11:35:25.406 INFO:teuthology.orchestra.run.smithi119.stdout:(20/20): containernetworking-plugins-1.1.1-3.mo 24 MB/s | 18 MB 00:00 2023-01-25T11:35:25.409 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:35:25.409 INFO:teuthology.orchestra.run.smithi119.stdout:Total 29 MB/s | 43 MB 00:01 2023-01-25T11:35:25.413 INFO:teuthology.orchestra.run.smithi157.stdout:(20/20): containernetworking-plugins-1.1.1-3.mo 25 MB/s | 18 MB 00:00 2023-01-25T11:35:25.415 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:35:25.416 INFO:teuthology.orchestra.run.smithi157.stdout:Total 30 MB/s | 43 MB 00:01 2023-01-25T11:35:25.830 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:35:25.841 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:35:25.875 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:35:25.875 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:35:25.886 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:35:25.886 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:35:26.499 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:35:26.503 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:35:26.514 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:35:26.518 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:35:27.563 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:35:27.576 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+b0 1/1 2023-01-25T11:35:27.617 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 1/28 2023-01-25T11:35:27.651 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:35:27.663 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+b0 1/1 2023-01-25T11:35:27.707 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 1/28 2023-01-25T11:35:27.814 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 1/28 2023-01-25T11:35:27.896 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 1/28 2023-01-25T11:35:45.296 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 1/28 2023-01-25T11:35:45.451 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 2/28 2023-01-25T11:35:45.499 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 1/28 2023-01-25T11:35:45.675 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 3/28 2023-01-25T11:35:45.700 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 2/28 2023-01-25T11:35:45.957 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 3/28 2023-01-25T11:35:46.183 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 4/28 2023-01-25T11:35:46.557 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 4/28 2023-01-25T11:35:47.940 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 5/28 2023-01-25T11:35:48.159 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : containernetworking-plugins-1:1.1.1-3.module_el8.7 6/28 2023-01-25T11:35:48.298 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 5/28 2023-01-25T11:35:48.348 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : yajl-2.1.0-11.el8.x86_64 7/28 2023-01-25T11:35:48.485 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 8/28 2023-01-25T11:35:48.524 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : containernetworking-plugins-1:1.1.1-3.module_el8.7 6/28 2023-01-25T11:35:48.726 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 9/28 2023-01-25T11:35:48.763 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : yajl-2.1.0-11.el8.x86_64 7/28 2023-01-25T11:35:48.950 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 8/28 2023-01-25T11:35:49.081 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : aardvark-dns-2:1.1.0-4.module_el8.7.0+1216+b022c01 10/28 2023-01-25T11:35:49.217 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 9/28 2023-01-25T11:35:49.585 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : aardvark-dns-2:1.1.0-4.module_el8.7.0+1216+b022c01 10/28 2023-01-25T11:35:49.937 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : netavark-2:1.1.0-6.module_el8.7.0+1216+b022c01d.x8 11/28 2023-01-25T11:35:50.221 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : buildah-1:1.27.0-2.module_el8.7.0+1216+b022c01d.x8 12/28 2023-01-25T11:35:50.489 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : netavark-2:1.1.0-6.module_el8.7.0+1216+b022c01d.x8 11/28 2023-01-25T11:35:50.502 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-podman-4.2.0-1.module_el8.7.0+1216+b022c01 13/28 2023-01-25T11:35:50.672 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : crun-1.5-1.module_el8.7.0+1216+b022c01d.x86_64 14/28 2023-01-25T11:35:50.785 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : buildah-1:1.27.0-2.module_el8.7.0+1216+b022c01d.x8 12/28 2023-01-25T11:35:50.894 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : udica-0.2.6-3.module_el8.7.0+1216+b022c01d.noarch 15/28 2023-01-25T11:35:51.043 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-podman-4.2.0-1.module_el8.7.0+1216+b022c01 13/28 2023-01-25T11:35:51.093 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d. 16/28 2023-01-25T11:35:51.112 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d. 16/28 2023-01-25T11:35:51.228 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : crun-1.5-1.module_el8.7.0+1216+b022c01d.x86_64 14/28 2023-01-25T11:35:51.480 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : udica-0.2.6-3.module_el8.7.0+1216+b022c01d.noarch 15/28 2023-01-25T11:35:51.524 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : conmon-3:2.1.4-1.module_el8.7.0+1216+b022c01d.x86_ 17/28 2023-01-25T11:35:51.708 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d. 16/28 2023-01-25T11:35:51.727 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d. 16/28 2023-01-25T11:35:52.114 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : conmon-3:2.1.4-1.module_el8.7.0+1216+b022c01d.x86_ 17/28 2023-01-25T11:35:52.295 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : toolbox-0.0.99.3-1.module_el8.5.0+877+1c30e0c9.x86 18/28 2023-01-25T11:35:52.520 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : skopeo-2:1.9.2-1.module_el8.7.0+1216+b022c01d.x86_ 19/28 2023-01-25T11:35:52.694 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cockpit-podman-53-1.module_el8.7.0+1216+b022c01d.n 20/28 2023-01-25T11:35:52.694 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : container-selinux-2:2.180.0-1.module_el8.7.0+1106+ 21/28 2023-01-25T11:35:52.886 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: container-selinux-2:2.180.0-1.module_el8.7.0+1106+ 21/28 2023-01-25T11:35:52.892 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : toolbox-0.0.99.3-1.module_el8.5.0+877+1c30e0c9.x86 18/28 2023-01-25T11:35:53.044 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : slirp4netns-1.1.8-2.module_el8.7.0+1106+45480ee0.x 22/28 2023-01-25T11:35:53.176 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : skopeo-2:1.9.2-1.module_el8.7.0+1216+b022c01d.x86_ 19/28 2023-01-25T11:35:53.193 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : runc-1:1.0.3-3.module_el8.7.0+1106+45480ee0.x86_64 23/28 2023-01-25T11:35:53.355 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : criu-3.15-3.module_el8.6.0+926+8bef8ae7.x86_64 24/28 2023-01-25T11:35:53.384 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cockpit-podman-53-1.module_el8.7.0+1216+b022c01d.n 20/28 2023-01-25T11:35:53.384 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : container-selinux-2:2.180.0-1.module_el8.7.0+1106+ 21/28 2023-01-25T11:35:53.502 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : libslirp-4.4.0-1.module_el8.6.0+926+8bef8ae7.x86_6 25/28 2023-01-25T11:35:53.593 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: container-selinux-2:2.180.0-1.module_el8.7.0+1106+ 21/28 2023-01-25T11:35:53.669 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : fuse-overlayfs-1.8.2-1.module_el8.7.0+1106+45480ee 26/28 2023-01-25T11:35:53.751 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : slirp4netns-1.1.8-2.module_el8.7.0+1106+45480ee0.x 22/28 2023-01-25T11:35:53.835 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : containernetworking-plugins-1:1.1.1-1.module_el8.7 27/28 2023-01-25T11:35:53.951 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : runc-1:1.0.3-3.module_el8.7.0+1106+45480ee0.x86_64 23/28 2023-01-25T11:35:53.993 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : conmon-2:2.1.0-1.module_el8.7.0+1106+45480ee0.x86_ 28/28 2023-01-25T11:35:54.119 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : criu-3.15-3.module_el8.6.0+926+8bef8ae7.x86_64 24/28 2023-01-25T11:35:54.300 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libslirp-4.4.0-1.module_el8.6.0+926+8bef8ae7.x86_6 25/28 2023-01-25T11:35:54.511 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : fuse-overlayfs-1.8.2-1.module_el8.7.0+1106+45480ee 26/28 2023-01-25T11:35:54.733 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : containernetworking-plugins-1:1.1.1-1.module_el8.7 27/28 2023-01-25T11:35:54.933 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : conmon-2:2.1.0-1.module_el8.7.0+1106+45480ee0.x86_ 28/28 2023-01-25T11:36:32.917 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 28/28 2023-01-25T11:36:33.439 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: conmon-2:2.1.0-1.module_el8.7.0+1106+45480ee0.x86_ 28/28 2023-01-25T11:36:33.439 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : aardvark-dns-2:1.1.0-4.module_el8.7.0+1216+b022c01 1/28 2023-01-25T11:36:33.439 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : buildah-1:1.27.0-2.module_el8.7.0+1216+b022c01d.x8 2/28 2023-01-25T11:36:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cockpit-podman-53-1.module_el8.7.0+1216+b022c01d.n 3/28 2023-01-25T11:36:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : crun-1.5-1.module_el8.7.0+1216+b022c01d.x86_64 4/28 2023-01-25T11:36:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : netavark-2:1.1.0-6.module_el8.7.0+1216+b022c01d.x8 5/28 2023-01-25T11:36:33.440 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-podman-4.2.0-1.module_el8.7.0+1216+b022c01 6/28 2023-01-25T11:36:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 7/28 2023-01-25T11:36:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 8/28 2023-01-25T11:36:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : skopeo-2:1.9.2-1.module_el8.7.0+1216+b022c01d.x86_ 9/28 2023-01-25T11:36:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : toolbox-0.0.99.3-1.module_el8.5.0+877+1c30e0c9.x86 10/28 2023-01-25T11:36:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : udica-0.2.6-3.module_el8.7.0+1216+b022c01d.noarch 11/28 2023-01-25T11:36:33.441 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : yajl-2.1.0-11.el8.x86_64 12/28 2023-01-25T11:36:33.442 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : conmon-3:2.1.4-1.module_el8.7.0+1216+b022c01d.x86_ 13/28 2023-01-25T11:36:33.442 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : conmon-2:2.1.0-1.module_el8.7.0+1106+45480ee0.x86_ 14/28 2023-01-25T11:36:33.442 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 15/28 2023-01-25T11:36:33.442 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : container-selinux-2:2.180.0-1.module_el8.7.0+1106+ 16/28 2023-01-25T11:36:33.442 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : containernetworking-plugins-1:1.1.1-3.module_el8.7 17/28 2023-01-25T11:36:33.443 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : containernetworking-plugins-1:1.1.1-1.module_el8.7 18/28 2023-01-25T11:36:33.443 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 19/28 2023-01-25T11:36:33.443 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : criu-3.15-3.module_el8.6.0+926+8bef8ae7.x86_64 20/28 2023-01-25T11:36:33.443 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d. 21/28 2023-01-25T11:36:33.443 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-overlayfs-1.8.2-1.module_el8.7.0+1106+45480ee 22/28 2023-01-25T11:36:33.444 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 23/28 2023-01-25T11:36:33.444 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libslirp-4.4.0-1.module_el8.6.0+926+8bef8ae7.x86_6 24/28 2023-01-25T11:36:33.444 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 25/28 2023-01-25T11:36:33.444 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : runc-1:1.0.3-3.module_el8.7.0+1106+45480ee0.x86_64 26/28 2023-01-25T11:36:33.444 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 27/28 2023-01-25T11:36:34.435 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : slirp4netns-1.1.8-2.module_el8.7.0+1106+45480ee0.x 28/28 2023-01-25T11:36:34.435 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:36:34.436 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-01-25T11:36:34.436 INFO:teuthology.orchestra.run.smithi157.stdout: conmon-3:2.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.436 INFO:teuthology.orchestra.run.smithi157.stdout: container-selinux-2:2.189.0-1.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:34.436 INFO:teuthology.orchestra.run.smithi157.stdout: containernetworking-plugins-1:1.1.1-3.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.436 INFO:teuthology.orchestra.run.smithi157.stdout: criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.437 INFO:teuthology.orchestra.run.smithi157.stdout: fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.437 INFO:teuthology.orchestra.run.smithi157.stdout: libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.437 INFO:teuthology.orchestra.run.smithi157.stdout: runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.437 INFO:teuthology.orchestra.run.smithi157.stdout: slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.437 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:36:34.438 INFO:teuthology.orchestra.run.smithi157.stdout: aardvark-dns-2:1.1.0-4.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.438 INFO:teuthology.orchestra.run.smithi157.stdout: buildah-1:1.27.0-2.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.438 INFO:teuthology.orchestra.run.smithi157.stdout: cockpit-podman-53-1.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:34.438 INFO:teuthology.orchestra.run.smithi157.stdout: crun-1.5-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.438 INFO:teuthology.orchestra.run.smithi157.stdout: netavark-2:1.1.0-6.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.439 INFO:teuthology.orchestra.run.smithi157.stdout: python3-podman-4.2.0-1.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:34.439 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-01-25T11:36:34.439 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-01-25T11:36:34.439 INFO:teuthology.orchestra.run.smithi157.stdout: skopeo-2:1.9.2-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:34.439 INFO:teuthology.orchestra.run.smithi157.stdout: toolbox-0.0.99.3-1.module_el8.5.0+877+1c30e0c9.x86_64 2023-01-25T11:36:34.440 INFO:teuthology.orchestra.run.smithi157.stdout: udica-0.2.6-3.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:34.440 INFO:teuthology.orchestra.run.smithi157.stdout: yajl-2.1.0-11.el8.x86_64 2023-01-25T11:36:34.440 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:36:34.440 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:36:34.764 DEBUG:teuthology.parallel:result is None 2023-01-25T11:36:35.233 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 28/28 2023-01-25T11:36:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: conmon-2:2.1.0-1.module_el8.7.0+1106+45480ee0.x86_ 28/28 2023-01-25T11:36:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : aardvark-dns-2:1.1.0-4.module_el8.7.0+1216+b022c01 1/28 2023-01-25T11:36:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : buildah-1:1.27.0-2.module_el8.7.0+1216+b022c01d.x8 2/28 2023-01-25T11:36:35.779 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cockpit-podman-53-1.module_el8.7.0+1216+b022c01d.n 3/28 2023-01-25T11:36:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : crun-1.5-1.module_el8.7.0+1216+b022c01d.x86_64 4/28 2023-01-25T11:36:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : netavark-2:1.1.0-6.module_el8.7.0+1216+b022c01d.x8 5/28 2023-01-25T11:36:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-podman-4.2.0-1.module_el8.7.0+1216+b022c01 6/28 2023-01-25T11:36:35.780 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 7/28 2023-01-25T11:36:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 8/28 2023-01-25T11:36:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : skopeo-2:1.9.2-1.module_el8.7.0+1216+b022c01d.x86_ 9/28 2023-01-25T11:36:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : toolbox-0.0.99.3-1.module_el8.5.0+877+1c30e0c9.x86 10/28 2023-01-25T11:36:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : udica-0.2.6-3.module_el8.7.0+1216+b022c01d.noarch 11/28 2023-01-25T11:36:35.781 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : yajl-2.1.0-11.el8.x86_64 12/28 2023-01-25T11:36:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : conmon-3:2.1.4-1.module_el8.7.0+1216+b022c01d.x86_ 13/28 2023-01-25T11:36:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : conmon-2:2.1.0-1.module_el8.7.0+1106+45480ee0.x86_ 14/28 2023-01-25T11:36:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : container-selinux-2:2.189.0-1.module_el8.7.0+1216+ 15/28 2023-01-25T11:36:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : container-selinux-2:2.180.0-1.module_el8.7.0+1106+ 16/28 2023-01-25T11:36:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : containernetworking-plugins-1:1.1.1-3.module_el8.7 17/28 2023-01-25T11:36:35.782 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : containernetworking-plugins-1:1.1.1-1.module_el8.7 18/28 2023-01-25T11:36:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 19/28 2023-01-25T11:36:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : criu-3.15-3.module_el8.6.0+926+8bef8ae7.x86_64 20/28 2023-01-25T11:36:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d. 21/28 2023-01-25T11:36:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-overlayfs-1.8.2-1.module_el8.7.0+1106+45480ee 22/28 2023-01-25T11:36:35.783 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_ 23/28 2023-01-25T11:36:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libslirp-4.4.0-1.module_el8.6.0+926+8bef8ae7.x86_6 24/28 2023-01-25T11:36:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 25/28 2023-01-25T11:36:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : runc-1:1.0.3-3.module_el8.7.0+1106+45480ee0.x86_64 26/28 2023-01-25T11:36:35.784 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x 27/28 2023-01-25T11:36:36.895 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : slirp4netns-1.1.8-2.module_el8.7.0+1106+45480ee0.x 28/28 2023-01-25T11:36:36.896 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:36:36.896 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-01-25T11:36:36.896 INFO:teuthology.orchestra.run.smithi119.stdout: conmon-3:2.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.896 INFO:teuthology.orchestra.run.smithi119.stdout: container-selinux-2:2.189.0-1.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:36.897 INFO:teuthology.orchestra.run.smithi119.stdout: containernetworking-plugins-1:1.1.1-3.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.897 INFO:teuthology.orchestra.run.smithi119.stdout: criu-3.15-3.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.897 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-overlayfs-1.9-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.898 INFO:teuthology.orchestra.run.smithi119.stdout: libslirp-4.4.0-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.898 INFO:teuthology.orchestra.run.smithi119.stdout: runc-1:1.1.4-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.898 INFO:teuthology.orchestra.run.smithi119.stdout: slirp4netns-1.2.0-2.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.898 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:36:36.898 INFO:teuthology.orchestra.run.smithi119.stdout: aardvark-dns-2:1.1.0-4.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.899 INFO:teuthology.orchestra.run.smithi119.stdout: buildah-1:1.27.0-2.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.899 INFO:teuthology.orchestra.run.smithi119.stdout: cockpit-podman-53-1.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:36.899 INFO:teuthology.orchestra.run.smithi119.stdout: crun-1.5-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.899 INFO:teuthology.orchestra.run.smithi119.stdout: netavark-2:1.1.0-6.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.899 INFO:teuthology.orchestra.run.smithi119.stdout: python3-podman-4.2.0-1.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:36.900 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-01-25T11:36:36.900 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-01-25T11:36:36.900 INFO:teuthology.orchestra.run.smithi119.stdout: skopeo-2:1.9.2-1.module_el8.7.0+1216+b022c01d.x86_64 2023-01-25T11:36:36.900 INFO:teuthology.orchestra.run.smithi119.stdout: toolbox-0.0.99.3-1.module_el8.5.0+877+1c30e0c9.x86_64 2023-01-25T11:36:36.901 INFO:teuthology.orchestra.run.smithi119.stdout: udica-0.2.6-3.module_el8.7.0+1216+b022c01d.noarch 2023-01-25T11:36:36.901 INFO:teuthology.orchestra.run.smithi119.stdout: yajl-2.1.0-11.el8.x86_64 2023-01-25T11:36:36.901 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:36:36.901 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:36:37.219 DEBUG:teuthology.parallel:result is None 2023-01-25T11:36:37.220 INFO:teuthology.run_tasks:Running task install... 2023-01-25T11:36:37.233 DEBUG:teuthology.task.install:project ceph 2023-01-25T11:36:37.233 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'}} 2023-01-25T11:36:37.234 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'} 2023-01-25T11:36:37.234 INFO:teuthology.task.install:Using flavor: default 2023-01-25T11:36:37.241 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T11:36:37.241 INFO:teuthology.task.install:extra packages: [] 2023-01-25T11:36:37.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo lsb_release -is 2023-01-25T11:36:37.280 INFO:teuthology.orchestra.run.smithi119.stdout:CentOSStream 2023-01-25T11:36:37.281 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T11:36:37.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo lsb_release -is 2023-01-25T11:36:37.282 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'tag': None, 'wait_for_package': False} 2023-01-25T11:36:37.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo lsb_release -is 2023-01-25T11:36:37.327 INFO:teuthology.orchestra.run.smithi157.stdout:CentOSStream 2023-01-25T11:36:37.329 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T11:36:37.329 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'tag': None, 'wait_for_package': False} 2023-01-25T11:36:37.330 DEBUG:teuthology.orchestra.run.smithi157:> sudo lsb_release -is 2023-01-25T11:36:37.358 INFO:teuthology.orchestra.run.smithi119.stdout:CentOSStream 2023-01-25T11:36:37.359 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T11:36:37.360 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:36:37.369 INFO:teuthology.orchestra.run.smithi157.stdout:CentOSStream 2023-01-25T11:36:37.371 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T11:36:37.371 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:36:37.514 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/ 2023-01-25T11:36:37.514 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1986.g44e4095d 2023-01-25T11:36:37.521 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/ 2023-01-25T11:36:37.521 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1986.g44e4095d 2023-01-25T11:36:37.629 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T11:36:37.630 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-01-25T11:36:37.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T11:36:37.634 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T11:36:37.635 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-01-25T11:36:37.635 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T11:36:37.657 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T11:36:37.658 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T11:36:37.663 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T11:36:37.664 DEBUG:teuthology.orchestra.run.smithi157:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T11:36:37.725 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T11:36:37.732 DEBUG:teuthology.orchestra.run.smithi157:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T11:36:37.802 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T11:36:37.810 DEBUG:teuthology.orchestra.run.smithi157:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T11:36:37.869 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2023-01-25T11:36:37.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2023-01-25T11:36:37.877 INFO:teuthology.orchestra.run.smithi157.stdout:check_obsoletes = 1 2023-01-25T11:36:37.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum clean all 2023-01-25T11:36:38.118 INFO:teuthology.orchestra.run.smithi119.stdout:61 files removed 2023-01-25T11:36:38.126 INFO:teuthology.orchestra.run.smithi157.stdout:61 files removed 2023-01-25T11:36:38.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw 2023-01-25T11:36:38.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-radosgw 2023-01-25T11:36:38.839 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 273 kB/s | 79 kB 00:00 2023-01-25T11:36:38.844 INFO:teuthology.orchestra.run.smithi157.stdout:ceph packages for x86_64 279 kB/s | 79 kB 00:00 2023-01-25T11:36:39.115 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 39 kB/s | 9.3 kB 00:00 2023-01-25T11:36:39.120 INFO:teuthology.orchestra.run.smithi157.stdout:ceph noarch packages 39 kB/s | 9.3 kB 00:00 2023-01-25T11:36:39.380 INFO:teuthology.orchestra.run.smithi157.stdout:ceph source packages 6.4 kB/s | 1.5 kB 00:00 2023-01-25T11:36:39.391 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 6.0 kB/s | 1.5 kB 00:00 2023-01-25T11:36:40.107 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - AppStream 39 MB/s | 27 MB 00:00 2023-01-25T11:36:40.285 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - AppStream 32 MB/s | 27 MB 00:00 2023-01-25T11:36:45.916 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - BaseOS 34 MB/s | 27 MB 00:00 2023-01-25T11:36:46.019 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - BaseOS 38 MB/s | 27 MB 00:00 2023-01-25T11:36:50.123 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - Extras 148 kB/s | 18 kB 00:00 2023-01-25T11:36:50.326 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - Extras 145 kB/s | 18 kB 00:00 2023-01-25T11:36:50.517 INFO:teuthology.orchestra.run.smithi157.stdout:CentOS-8 - PowerTools 21 MB/s | 5.6 MB 00:00 2023-01-25T11:36:50.744 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-8 - PowerTools 20 MB/s | 5.6 MB 00:00 2023-01-25T11:36:51.732 INFO:teuthology.orchestra.run.smithi157.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-01-25T11:36:51.985 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-01-25T11:36:53.066 INFO:teuthology.orchestra.run.smithi157.stdout:Extra Packages for Enterprise Linux 11 MB/s | 13 MB 00:01 2023-01-25T11:36:53.344 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 11 MB/s | 13 MB 00:01 2023-01-25T11:36:56.608 INFO:teuthology.orchestra.run.smithi157.stdout:lab-extras 70 kB/s | 24 kB 00:00 2023-01-25T11:36:56.703 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 396 kB/s | 24 kB 00:00 2023-01-25T11:37:00.019 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:37:00.022 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:37:00.022 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:37:00.023 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:37:00.023 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:37:00.023 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 13 M 2023-01-25T11:37:00.023 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrading: 2023-01-25T11:37:00.023 INFO:teuthology.orchestra.run.smithi157.stdout: librados2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T11:37:00.024 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T11:37:00.024 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:37:00.024 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 4.8 M 2023-01-25T11:37:00.024 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 21 M 2023-01-25T11:37:00.025 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 24 k 2023-01-25T11:37:00.025 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T11:37:00.025 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 767 k 2023-01-25T11:37:00.025 INFO:teuthology.orchestra.run.smithi157.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T11:37:00.026 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T11:37:00.026 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 474 k 2023-01-25T11:37:00.026 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T11:37:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.5 M 2023-01-25T11:37:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T11:37:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T11:37:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse 2023-01-25T11:37:00.027 INFO:teuthology.orchestra.run.smithi157.stdout: x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 k 2023-01-25T11:37:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 115 k 2023-01-25T11:37:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 190 k 2023-01-25T11:37:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T11:37:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 372 k 2023-01-25T11:37:00.028 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 364 k 2023-01-25T11:37:00.029 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 106 k 2023-01-25T11:37:00.029 INFO:teuthology.orchestra.run.smithi157.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T11:37:00.029 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:00.029 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:37:00.029 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:37:00.030 INFO:teuthology.orchestra.run.smithi157.stdout:Install 21 Packages 2023-01-25T11:37:00.030 INFO:teuthology.orchestra.run.smithi157.stdout:Upgrade 2 Packages 2023-01-25T11:37:00.030 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:00.031 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 58 M 2023-01-25T11:37:00.031 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:37:00.176 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:37:00.178 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:37:00.178 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:37:00.178 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:37:00.179 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:37:00.179 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 13 M 2023-01-25T11:37:00.179 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-01-25T11:37:00.179 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T11:37:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.6 M 2023-01-25T11:37:00.180 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:37:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 4.8 M 2023-01-25T11:37:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 21 M 2023-01-25T11:37:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 24 k 2023-01-25T11:37:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T11:37:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 767 k 2023-01-25T11:37:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T11:37:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2023-01-25T11:37:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 474 k 2023-01-25T11:37:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2023-01-25T11:37:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.5 M 2023-01-25T11:37:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T11:37:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2023-01-25T11:37:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse 2023-01-25T11:37:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 k 2023-01-25T11:37:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 115 k 2023-01-25T11:37:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 190 k 2023-01-25T11:37:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2023-01-25T11:37:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 372 k 2023-01-25T11:37:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 364 k 2023-01-25T11:37:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 106 k 2023-01-25T11:37:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T11:37:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:00.184 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:37:00.185 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:37:00.185 INFO:teuthology.orchestra.run.smithi119.stdout:Install 21 Packages 2023-01-25T11:37:00.185 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 2 Packages 2023-01-25T11:37:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:00.186 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 58 M 2023-01-25T11:37:00.186 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:37:00.634 INFO:teuthology.orchestra.run.smithi157.stdout:(1/23): ceph-base-18.0.0-1986.g44e4095d.el8.x86 8.0 MB/s | 4.8 MB 00:00 2023-01-25T11:37:00.693 INFO:teuthology.orchestra.run.smithi157.stdout:(2/23): ceph-selinux-18.0.0-1986.g44e4095d.el8. 418 kB/s | 24 kB 00:00 2023-01-25T11:37:00.794 INFO:teuthology.orchestra.run.smithi157.stdout:(3/23): libcephfs2-18.0.0-1986.g44e4095d.el8.x8 7.5 MB/s | 767 kB 00:00 2023-01-25T11:37:00.894 INFO:teuthology.orchestra.run.smithi157.stdout:(4/23): libradosstriper1-18.0.0-1986.g44e4095d. 4.6 MB/s | 474 kB 00:00 2023-01-25T11:37:01.536 INFO:teuthology.orchestra.run.smithi157.stdout:(5/23): librgw2-18.0.0-1986.g44e4095d.el8.x86_6 10 MB/s | 6.5 MB 00:00 2023-01-25T11:37:01.603 INFO:teuthology.orchestra.run.smithi157.stdout:(6/23): python3-ceph-argparse-18.0.0-1986.g44e4 669 kB/s | 45 kB 00:00 2023-01-25T11:37:01.669 INFO:teuthology.orchestra.run.smithi157.stdout:(7/23): python3-ceph-common-18.0.0-1986.g44e409 1.7 MB/s | 115 kB 00:00 2023-01-25T11:37:01.745 INFO:teuthology.orchestra.run.smithi157.stdout:(8/23): python3-cephfs-18.0.0-1986.g44e4095d.el 2.5 MB/s | 190 kB 00:00 2023-01-25T11:37:01.828 INFO:teuthology.orchestra.run.smithi157.stdout:(9/23): python3-rados-18.0.0-1986.g44e4095d.el8 4.4 MB/s | 372 kB 00:00 2023-01-25T11:37:01.912 INFO:teuthology.orchestra.run.smithi157.stdout:(10/23): python3-rbd-18.0.0-1986.g44e4095d.el8. 4.3 MB/s | 364 kB 00:00 2023-01-25T11:37:01.979 INFO:teuthology.orchestra.run.smithi157.stdout:(11/23): python3-rgw-18.0.0-1986.g44e4095d.el8. 1.6 MB/s | 106 kB 00:00 2023-01-25T11:37:02.187 INFO:teuthology.orchestra.run.smithi157.stdout:(12/23): ceph-common-18.0.0-1986.g44e4095d.el8. 9.9 MB/s | 21 MB 00:02 2023-01-25T11:37:02.321 INFO:teuthology.orchestra.run.smithi157.stdout:(13/23): ceph-radosgw-18.0.0-1986.g44e4095d.el8 5.7 MB/s | 13 MB 00:02 2023-01-25T11:37:02.363 INFO:teuthology.orchestra.run.smithi157.stdout:(14/23): librdkafka-0.11.4-3.el8.x86_64.rpm 922 kB/s | 354 kB 00:00 2023-01-25T11:37:02.396 INFO:teuthology.orchestra.run.smithi157.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.6 MB/s | 47 kB 00:00 2023-01-25T11:37:02.414 INFO:teuthology.orchestra.run.smithi157.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 512 kB/s | 44 kB 00:00 2023-01-25T11:37:02.439 INFO:teuthology.orchestra.run.smithi157.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.0 MB/s | 259 kB 00:00 2023-01-25T11:37:02.781 INFO:teuthology.orchestra.run.smithi157.stdout:(18/23): gperftools-libs-2.7-9.el8.x86_64.rpm 797 kB/s | 306 kB 00:00 2023-01-25T11:37:02.806 INFO:teuthology.orchestra.run.smithi157.stdout:(19/23): liboath-2.6.2-3.el8.x86_64.rpm 150 kB/s | 59 kB 00:00 2023-01-25T11:37:02.990 INFO:teuthology.orchestra.run.smithi157.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 8.3 MB/s | 1.7 MB 00:00 2023-01-25T11:37:03.015 INFO:teuthology.orchestra.run.smithi157.stdout:(21/23): libunwind-1.3.1-3.el8.x86_64.rpm 130 kB/s | 75 kB 00:00 2023-01-25T11:37:03.082 INFO:teuthology.orchestra.run.smithi157.stdout:(22/23): librados2-18.0.0-1986.g44e4095d.el8.x8 13 MB/s | 3.6 MB 00:00 2023-01-25T11:37:03.315 INFO:teuthology.orchestra.run.smithi157.stdout:(23/23): librbd1-18.0.0-1986.g44e4095d.el8.x86_ 11 MB/s | 3.6 MB 00:00 2023-01-25T11:37:03.318 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:37:03.319 INFO:teuthology.orchestra.run.smithi157.stdout:Total 18 MB/s | 58 MB 00:03 2023-01-25T11:37:03.358 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:37:03.405 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:37:03.405 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:37:03.463 INFO:teuthology.orchestra.run.smithi119.stdout:(1/23): ceph-base-18.0.0-1986.g44e4095d.el8.x86 1.5 MB/s | 4.8 MB 00:03 2023-01-25T11:37:03.543 INFO:teuthology.orchestra.run.smithi119.stdout:(2/23): ceph-selinux-18.0.0-1986.g44e4095d.el8. 302 kB/s | 24 kB 00:00 2023-01-25T11:37:03.760 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:37:03.764 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:37:03.788 INFO:teuthology.orchestra.run.smithi119.stdout:(3/23): libcephfs2-18.0.0-1986.g44e4095d.el8.x8 3.1 MB/s | 767 kB 00:00 2023-01-25T11:37:03.930 INFO:teuthology.orchestra.run.smithi119.stdout:(4/23): libradosstriper1-18.0.0-1986.g44e4095d. 3.3 MB/s | 474 kB 00:00 2023-01-25T11:37:04.940 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:37:05.120 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T11:37:05.305 INFO:teuthology.orchestra.run.smithi119.stdout:(5/23): librgw2-18.0.0-1986.g44e4095d.el8.x86_6 4.7 MB/s | 6.5 MB 00:01 2023-01-25T11:37:05.348 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T11:37:05.377 INFO:teuthology.orchestra.run.smithi119.stdout:(6/23): python3-ceph-argparse-18.0.0-1986.g44e4 616 kB/s | 45 kB 00:00 2023-01-25T11:37:05.430 INFO:teuthology.orchestra.run.smithi119.stdout:(7/23): python3-ceph-common-18.0.0-1986.g44e409 2.1 MB/s | 115 kB 00:00 2023-01-25T11:37:05.505 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T11:37:05.522 INFO:teuthology.orchestra.run.smithi119.stdout:(8/23): python3-cephfs-18.0.0-1986.g44e4095d.el 2.0 MB/s | 190 kB 00:00 2023-01-25T11:37:05.689 INFO:teuthology.orchestra.run.smithi119.stdout:(9/23): python3-rados-18.0.0-1986.g44e4095d.el8 2.2 MB/s | 372 kB 00:00 2023-01-25T11:37:05.848 INFO:teuthology.orchestra.run.smithi119.stdout:(10/23): python3-rbd-18.0.0-1986.g44e4095d.el8. 2.2 MB/s | 364 kB 00:00 2023-01-25T11:37:05.907 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T11:37:05.914 INFO:teuthology.orchestra.run.smithi119.stdout:(11/23): python3-rgw-18.0.0-1986.g44e4095d.el8. 1.6 MB/s | 106 kB 00:00 2023-01-25T11:37:06.001 INFO:teuthology.orchestra.run.smithi119.stdout:(12/23): librdkafka-0.11.4-3.el8.x86_64.rpm 4.0 MB/s | 354 kB 00:00 2023-01-25T11:37:06.032 INFO:teuthology.orchestra.run.smithi119.stdout:(13/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 8.1 MB/s | 259 kB 00:00 2023-01-25T11:37:06.066 INFO:teuthology.orchestra.run.smithi119.stdout:(14/23): python3-prettytable-0.7.2-14.el8.noarc 1.3 MB/s | 44 kB 00:00 2023-01-25T11:37:06.091 INFO:teuthology.orchestra.run.smithi119.stdout:(15/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.8 MB/s | 47 kB 00:00 2023-01-25T11:37:06.114 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T11:37:06.208 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T11:37:06.312 INFO:teuthology.orchestra.run.smithi119.stdout:(16/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2023-01-25T11:37:06.442 INFO:teuthology.orchestra.run.smithi119.stdout:(17/23): ceph-radosgw-18.0.0-1986.g44e4095d.el8 2.1 MB/s | 13 MB 00:06 2023-01-25T11:37:06.467 INFO:teuthology.orchestra.run.smithi119.stdout:(18/23): liboath-2.6.2-3.el8.x86_64.rpm 379 kB/s | 59 kB 00:00 2023-01-25T11:37:06.503 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T11:37:06.584 INFO:teuthology.orchestra.run.smithi119.stdout:(19/23): libunwind-1.3.1-3.el8.x86_64.rpm 529 kB/s | 75 kB 00:00 2023-01-25T11:37:06.655 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T11:37:06.667 INFO:teuthology.orchestra.run.smithi119.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 8.7 MB/s | 1.7 MB 00:00 2023-01-25T11:37:07.048 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T11:37:07.077 INFO:teuthology.orchestra.run.smithi119.stdout:(21/23): ceph-common-18.0.0-1986.g44e4095d.el8. 3.1 MB/s | 21 MB 00:06 2023-01-25T11:37:07.205 INFO:teuthology.orchestra.run.smithi157.stdout: Upgrading : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T11:37:07.254 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T11:37:07.428 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T11:37:07.435 INFO:teuthology.orchestra.run.smithi119.stdout:(22/23): librados2-18.0.0-1986.g44e4095d.el8.x8 4.3 MB/s | 3.6 MB 00:00 2023-01-25T11:37:07.494 INFO:teuthology.orchestra.run.smithi119.stdout:(23/23): librbd1-18.0.0-1986.g44e4095d.el8.x86_ 4.3 MB/s | 3.6 MB 00:00 2023-01-25T11:37:07.499 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:37:07.499 INFO:teuthology.orchestra.run.smithi119.stdout:Total 8.0 MB/s | 58 MB 00:07 2023-01-25T11:37:07.538 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:37:07.564 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T11:37:07.583 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:37:07.584 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:37:07.657 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T11:37:07.782 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 10/25 2023-01-25T11:37:07.939 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:37:07.943 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:37:07.982 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T11:37:08.197 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T11:37:08.363 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T11:37:08.429 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T11:37:08.655 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T11:37:09.149 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:37:09.343 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T11:37:09.432 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T11:37:09.572 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/25 2023-01-25T11:37:09.596 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T11:37:09.667 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T11:37:09.770 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T11:37:09.846 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T11:37:10.030 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T11:37:10.171 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/25 2023-01-25T11:37:10.239 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 19/25 2023-01-25T11:37:10.420 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T11:37:10.527 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T11:37:10.809 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T11:37:11.020 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T11:37:11.414 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 6/25 2023-01-25T11:37:11.612 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T11:37:11.674 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T11:37:11.884 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-25T11:37:12.062 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T11:37:12.127 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/25 2023-01-25T11:37:12.297 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 10/25 2023-01-25T11:37:12.538 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T11:37:12.762 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T11:37:12.912 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T11:37:12.982 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 13/25 2023-01-25T11:37:13.219 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 14/25 2023-01-25T11:37:13.759 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T11:37:13.979 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T11:37:14.015 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 15/25 2023-01-25T11:37:14.195 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T11:37:14.271 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 16/25 2023-01-25T11:37:14.511 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 17/25 2023-01-25T11:37:14.515 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T11:37:14.737 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 18/25 2023-01-25T11:37:14.824 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T11:37:14.933 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T11:37:14.987 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 19/25 2023-01-25T11:37:15.121 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T11:37:18.528 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T11:37:18.795 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T11:37:19.332 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 20/25 2023-01-25T11:37:19.688 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T11:37:19.788 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 21/25 2023-01-25T11:37:20.002 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T11:37:34.282 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T11:37:34.283 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /sys 2023-01-25T11:37:34.283 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /proc 2023-01-25T11:37:34.283 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-01-25T11:37:34.283 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /run 2023-01-25T11:37:34.284 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /mnt 2023-01-25T11:37:34.284 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /var/tmp 2023-01-25T11:37:34.284 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /home 2023-01-25T11:37:34.284 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /tmp 2023-01-25T11:37:34.284 INFO:teuthology.orchestra.run.smithi157.stdout:skipping the directory /dev 2023-01-25T11:37:34.285 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:35.679 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T11:37:35.700 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T11:37:35.701 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T11:37:35.936 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T11:37:35.936 INFO:teuthology.orchestra.run.smithi157.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T11:37:36.659 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T11:37:36.659 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 1/25 2023-01-25T11:37:36.659 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2/25 2023-01-25T11:37:36.659 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T11:37:36.660 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T11:37:36.660 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T11:37:36.660 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 6/25 2023-01-25T11:37:36.660 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T11:37:36.661 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 8/25 2023-01-25T11:37:36.661 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 9/25 2023-01-25T11:37:36.661 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 10/25 2023-01-25T11:37:36.661 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T11:37:36.661 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T11:37:36.661 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 13/25 2023-01-25T11:37:36.662 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T11:37:36.662 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T11:37:36.662 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T11:37:36.662 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T11:37:36.662 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T11:37:36.663 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T11:37:36.663 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T11:37:36.663 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T11:37:36.663 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T11:37:36.663 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T11:37:36.664 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 24/25 2023-01-25T11:37:37.483 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T11:37:37.483 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:37.483 INFO:teuthology.orchestra.run.smithi157.stdout:Upgraded: 2023-01-25T11:37:37.483 INFO:teuthology.orchestra.run.smithi157.stdout: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.484 INFO:teuthology.orchestra.run.smithi157.stdout: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.484 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:37:37.484 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.484 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.484 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.485 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.485 INFO:teuthology.orchestra.run.smithi157.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T11:37:37.485 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.485 INFO:teuthology.orchestra.run.smithi157.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T11:37:37.485 INFO:teuthology.orchestra.run.smithi157.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T11:37:37.486 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.486 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T11:37:37.486 INFO:teuthology.orchestra.run.smithi157.stdout: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.486 INFO:teuthology.orchestra.run.smithi157.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T11:37:37.486 INFO:teuthology.orchestra.run.smithi157.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T11:37:37.486 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.487 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.487 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.487 INFO:teuthology.orchestra.run.smithi157.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T11:37:37.487 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.487 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.488 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:37.488 INFO:teuthology.orchestra.run.smithi157.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T11:37:37.488 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:37.488 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:37:37.763 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-test 2023-01-25T11:37:38.498 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:42 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:37:39.645 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:37:39.646 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:37:39.647 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:37:39.647 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:37:39.647 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:37:39.647 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 M 2023-01-25T11:37:39.648 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:37:39.648 INFO:teuthology.orchestra.run.smithi157.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T11:37:39.648 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 137 k 2023-01-25T11:37:39.648 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T11:37:39.649 INFO:teuthology.orchestra.run.smithi157.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T11:37:39.649 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T11:37:39.649 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:39.649 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:37:39.649 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:37:39.649 INFO:teuthology.orchestra.run.smithi157.stdout:Install 6 Packages 2023-01-25T11:37:39.650 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:39.650 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 46 M 2023-01-25T11:37:39.650 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 204 M 2023-01-25T11:37:39.651 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:37:39.748 INFO:teuthology.orchestra.run.smithi157.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2023-01-25T11:37:39.767 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T11:37:39.767 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2023-01-25T11:37:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2023-01-25T11:37:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-01-25T11:37:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /run 2023-01-25T11:37:39.768 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2023-01-25T11:37:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2023-01-25T11:37:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2023-01-25T11:37:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2023-01-25T11:37:39.769 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-01-25T11:37:39.769 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:39.773 INFO:teuthology.orchestra.run.smithi157.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.6 MB/s | 187 kB 00:00 2023-01-25T11:37:39.799 INFO:teuthology.orchestra.run.smithi157.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 13 MB/s | 323 kB 00:00 2023-01-25T11:37:39.824 INFO:teuthology.orchestra.run.smithi157.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2023-01-25T11:37:39.891 INFO:teuthology.orchestra.run.smithi157.stdout:(5/6): libcephsqlite-18.0.0-1986.g44e4095d.el8. 568 kB/s | 137 kB 00:00 2023-01-25T11:37:41.176 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T11:37:41.196 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 23/25 2023-01-25T11:37:41.197 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T11:37:41.451 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T11:37:42.141 INFO:teuthology.orchestra.run.smithi157.stdout:(6/6): ceph-test-18.0.0-1986.g44e4095d.el8.x86_ 18 MB/s | 45 MB 00:02 2023-01-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:37:42.144 INFO:teuthology.orchestra.run.smithi157.stdout:Total 18 MB/s | 46 MB 00:02 2023-01-25T11:37:42.189 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:37:42.210 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:37:42.210 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:37:42.218 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T11:37:42.219 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 1/25 2023-01-25T11:37:42.219 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2/25 2023-01-25T11:37:42.219 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 3/25 2023-01-25T11:37:42.219 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 4/25 2023-01-25T11:37:42.219 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 5/25 2023-01-25T11:37:42.220 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_6 6/25 2023-01-25T11:37:42.220 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 7/25 2023-01-25T11:37:42.220 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8. 8/25 2023-01-25T11:37:42.220 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x8 9/25 2023-01-25T11:37:42.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 10/25 2023-01-25T11:37:42.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 11/25 2023-01-25T11:37:42.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 12/25 2023-01-25T11:37:42.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 13/25 2023-01-25T11:37:42.221 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 14/25 2023-01-25T11:37:42.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 15/25 2023-01-25T11:37:42.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-25T11:37:42.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 17/25 2023-01-25T11:37:42.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-25T11:37:42.222 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 19/25 2023-01-25T11:37:42.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 20/25 2023-01-25T11:37:42.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 21/25 2023-01-25T11:37:42.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 22/25 2023-01-25T11:37:42.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T11:37:42.223 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 24/25 2023-01-25T11:37:42.480 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:37:42.484 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:37:42.933 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:37:43.091 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T11:37:43.092 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:43.092 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-01-25T11:37:43.092 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.092 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.093 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:37:43.093 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.093 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.093 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.093 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.094 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T11:37:43.094 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.094 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T11:37:43.094 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T11:37:43.094 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.095 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T11:37:43.095 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.095 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T11:37:43.095 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T11:37:43.095 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.096 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.096 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.096 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T11:37:43.096 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.096 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.097 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:43.097 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T11:37:43.097 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:43.097 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:37:43.110 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T11:37:43.327 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T11:37:43.406 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-test 2023-01-25T11:37:43.474 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T11:37:43.554 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T11:37:43.736 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T11:37:43.899 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T11:37:44.131 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:00:48 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:37:45.279 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:37:45.280 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:37:45.280 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:37:45.280 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:37:45.280 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:37:45.281 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 45 M 2023-01-25T11:37:45.281 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:37:45.281 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2023-01-25T11:37:45.281 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 137 k 2023-01-25T11:37:45.281 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2023-01-25T11:37:45.282 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2023-01-25T11:37:45.282 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2023-01-25T11:37:45.282 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:45.282 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:37:45.282 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:37:45.283 INFO:teuthology.orchestra.run.smithi119.stdout:Install 6 Packages 2023-01-25T11:37:45.283 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:45.284 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 46 M 2023-01-25T11:37:45.284 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 204 M 2023-01-25T11:37:45.284 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:37:45.347 INFO:teuthology.orchestra.run.smithi119.stdout:(1/6): jq-1.6-6.el8.x86_64.rpm 3.1 MB/s | 202 kB 00:00 2023-01-25T11:37:45.380 INFO:teuthology.orchestra.run.smithi119.stdout:(2/6): oniguruma-6.8.2-2.el8.x86_64.rpm 7.2 MB/s | 187 kB 00:00 2023-01-25T11:37:45.414 INFO:teuthology.orchestra.run.smithi119.stdout:(3/6): socat-1.7.4.1-1.el8.x86_64.rpm 9.7 MB/s | 323 kB 00:00 2023-01-25T11:37:45.439 INFO:teuthology.orchestra.run.smithi119.stdout:(4/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.8 MB/s | 71 kB 00:00 2023-01-25T11:37:45.556 INFO:teuthology.orchestra.run.smithi119.stdout:(5/6): libcephsqlite-18.0.0-1986.g44e4095d.el8. 501 kB/s | 137 kB 00:00 2023-01-25T11:37:47.748 INFO:teuthology.orchestra.run.smithi119.stdout:(6/6): ceph-test-18.0.0-1986.g44e4095d.el8.x86_ 18 MB/s | 45 MB 00:02 2023-01-25T11:37:47.751 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:37:47.751 INFO:teuthology.orchestra.run.smithi119.stdout:Total 19 MB/s | 46 MB 00:02 2023-01-25T11:37:47.793 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:37:47.814 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:37:47.814 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:37:48.086 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:37:48.089 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:37:48.221 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T11:37:48.495 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T11:37:48.547 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:37:48.792 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 1/6 2023-01-25T11:37:48.794 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T11:37:48.795 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 1/6 2023-01-25T11:37:48.795 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2/6 2023-01-25T11:37:48.795 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T11:37:48.795 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T11:37:48.796 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T11:37:49.025 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 2/6 2023-01-25T11:37:49.180 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T11:37:49.294 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 3/6 2023-01-25T11:37:49.331 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T11:37:49.331 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:49.331 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:37:49.331 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:49.332 INFO:teuthology.orchestra.run.smithi157.stdout: jq-1.6-6.el8.x86_64 2023-01-25T11:37:49.332 INFO:teuthology.orchestra.run.smithi157.stdout: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:49.332 INFO:teuthology.orchestra.run.smithi157.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T11:37:49.332 INFO:teuthology.orchestra.run.smithi157.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T11:37:49.333 INFO:teuthology.orchestra.run.smithi157.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T11:37:49.333 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:49.333 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:37:49.476 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-6.el8.x86_64 4/6 2023-01-25T11:37:49.525 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph 2023-01-25T11:37:49.689 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T11:37:50.253 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:00:54 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:37:51.411 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:37:51.414 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2023-01-25T11:37:51.415 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:37:51.415 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2023-01-25T11:37:51.415 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:37:51.415 INFO:teuthology.orchestra.run.smithi157.stdout: ceph x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.4 k 2023-01-25T11:37:51.415 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:37:51.416 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 2.3 M 2023-01-25T11:37:51.416 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 1.5 M 2023-01-25T11:37:51.416 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 236 k 2023-01-25T11:37:51.416 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.9 M 2023-01-25T11:37:51.417 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 17 M 2023-01-25T11:37:51.417 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T11:37:51.417 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T11:37:51.417 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T11:37:51.418 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T11:37:51.418 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T11:37:51.418 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T11:37:51.418 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T11:37:51.419 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T11:37:51.419 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T11:37:51.419 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T11:37:51.419 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T11:37:51.419 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T11:37:51.420 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T11:37:51.420 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T11:37:51.420 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T11:37:51.420 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T11:37:51.421 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T11:37:51.421 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T11:37:51.421 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T11:37:51.421 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T11:37:51.421 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T11:37:51.422 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T11:37:51.422 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T11:37:51.422 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T11:37:51.422 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T11:37:51.422 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T11:37:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:51.423 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:37:51.423 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================= 2023-01-25T11:37:51.423 INFO:teuthology.orchestra.run.smithi157.stdout:Install 32 Packages 2023-01-25T11:37:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:37:51.424 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 28 M 2023-01-25T11:37:51.424 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 107 M 2023-01-25T11:37:51.425 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:37:51.561 INFO:teuthology.orchestra.run.smithi157.stdout:(1/32): ceph-18.0.0-1986.g44e4095d.el8.x86_64.r 46 kB/s | 6.4 kB 00:00 2023-01-25T11:37:51.776 INFO:teuthology.orchestra.run.smithi157.stdout:(2/32): ceph-mgr-18.0.0-1986.g44e4095d.el8.x86_ 4.3 MB/s | 1.5 MB 00:00 2023-01-25T11:37:51.828 INFO:teuthology.orchestra.run.smithi157.stdout:(3/32): ceph-mds-18.0.0-1986.g44e4095d.el8.x86_ 5.6 MB/s | 2.3 MB 00:00 2023-01-25T11:37:51.911 INFO:teuthology.orchestra.run.smithi157.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1986.g44e4 2.8 MB/s | 236 kB 00:00 2023-01-25T11:37:51.971 INFO:teuthology.orchestra.run.smithi157.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T11:37:51.995 INFO:teuthology.orchestra.run.smithi157.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 1.5 MB/s | 39 kB 00:00 2023-01-25T11:37:52.021 INFO:teuthology.orchestra.run.smithi157.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 4.0 MB/s | 103 kB 00:00 2023-01-25T11:37:52.046 INFO:teuthology.orchestra.run.smithi157.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-01-25T11:37:52.080 INFO:teuthology.orchestra.run.smithi157.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-01-25T11:37:52.105 INFO:teuthology.orchestra.run.smithi157.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2023-01-25T11:37:52.139 INFO:teuthology.orchestra.run.smithi157.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-25T11:37:52.164 INFO:teuthology.orchestra.run.smithi157.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-25T11:37:52.264 INFO:teuthology.orchestra.run.smithi157.stdout:(13/32): ceph-mon-18.0.0-1986.g44e4095d.el8.x86 5.5 MB/s | 3.9 MB 00:00 2023-01-25T11:37:52.298 INFO:teuthology.orchestra.run.smithi157.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 326 kB/s | 44 kB 00:00 2023-01-25T11:37:52.398 INFO:teuthology.orchestra.run.smithi157.stdout:(15/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.7 MB/s | 173 kB 00:00 2023-01-25T11:37:52.432 INFO:teuthology.orchestra.run.smithi157.stdout:(16/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.1 MB/s | 185 kB 00:00 2023-01-25T11:37:52.491 INFO:teuthology.orchestra.run.smithi157.stdout:(17/32): python3-cherrypy-18.4.0-1.el8.noarch.r 4.1 MB/s | 384 kB 00:00 2023-01-25T11:37:52.516 INFO:teuthology.orchestra.run.smithi157.stdout:(18/32): python3-jaraco-6.2-6.el8.noarch.rpm 132 kB/s | 11 kB 00:00 2023-01-25T11:37:52.541 INFO:teuthology.orchestra.run.smithi157.stdout:(19/32): python3-jaraco-functools-2.0-4.el8.noa 348 kB/s | 18 kB 00:00 2023-01-25T11:37:52.567 INFO:teuthology.orchestra.run.smithi157.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 969 kB/s | 49 kB 00:00 2023-01-25T11:37:52.592 INFO:teuthology.orchestra.run.smithi157.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2023-01-25T11:37:52.759 INFO:teuthology.orchestra.run.smithi157.stdout:(22/32): ceph-osd-18.0.0-1986.g44e4095d.el8.x86 17 MB/s | 17 MB 00:00 2023-01-25T11:37:52.784 INFO:teuthology.orchestra.run.smithi157.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 81 kB/s | 16 kB 00:00 2023-01-25T11:37:52.818 INFO:teuthology.orchestra.run.smithi157.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-01-25T11:37:52.843 INFO:teuthology.orchestra.run.smithi157.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 409 kB/s | 24 kB 00:00 2023-01-25T11:37:52.868 INFO:teuthology.orchestra.run.smithi157.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 581 kB/s | 29 kB 00:00 2023-01-25T11:37:52.894 INFO:teuthology.orchestra.run.smithi157.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 545 kB/s | 27 kB 00:00 2023-01-25T11:37:52.919 INFO:teuthology.orchestra.run.smithi157.stdout:(28/32): python3-simplegeneric-0.8.1-17.el8.noa 117 kB/s | 19 kB 00:00 2023-01-25T11:37:52.944 INFO:teuthology.orchestra.run.smithi157.stdout:(29/32): python3-waitress-1.2.1-2.el8.1.noarch. 3.1 MB/s | 241 kB 00:00 2023-01-25T11:37:52.995 INFO:teuthology.orchestra.run.smithi157.stdout:(30/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.4 MB/s | 251 kB 00:00 2023-01-25T11:37:53.020 INFO:teuthology.orchestra.run.smithi157.stdout:(31/32): python3-zc-lockfile-2.0-2.el8.noarch.r 304 kB/s | 23 kB 00:00 2023-01-25T11:37:53.045 INFO:teuthology.orchestra.run.smithi157.stdout:(32/32): python3-webtest-2.0.33-1.el8.noarch.rp 679 kB/s | 85 kB 00:00 2023-01-25T11:37:53.048 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:37:53.048 INFO:teuthology.orchestra.run.smithi157.stdout:Total 17 MB/s | 28 MB 00:01 2023-01-25T11:37:53.126 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:37:53.163 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:37:53.163 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:37:53.392 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:37:53.398 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:37:54.010 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 5/6 2023-01-25T11:37:54.318 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T11:37:54.599 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:37:54.618 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 6/6 2023-01-25T11:37:54.618 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 1/6 2023-01-25T11:37:54.619 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2/6 2023-01-25T11:37:54.619 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-6.el8.x86_64 3/6 2023-01-25T11:37:54.619 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/6 2023-01-25T11:37:54.619 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 5/6 2023-01-25T11:37:54.853 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T11:37:55.105 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T11:37:55.154 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 6/6 2023-01-25T11:37:55.155 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:55.155 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:37:55.156 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:55.156 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-6.el8.x86_64 2023-01-25T11:37:55.156 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:37:55.156 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T11:37:55.157 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T11:37:55.157 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T11:37:55.157 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:55.157 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:37:55.360 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T11:37:55.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph 2023-01-25T11:37:55.645 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T11:37:55.843 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T11:37:56.015 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T11:37:56.126 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:00 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:37:56.216 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T11:37:56.394 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T11:37:56.594 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T11:37:56.873 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T11:37:57.136 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T11:37:57.271 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:37:57.274 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2023-01-25T11:37:57.275 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:37:57.275 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2023-01-25T11:37:57.275 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:37:57.275 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 6.4 k 2023-01-25T11:37:57.275 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:37:57.276 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 2.3 M 2023-01-25T11:37:57.276 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 1.5 M 2023-01-25T11:37:57.276 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 236 k 2023-01-25T11:37:57.276 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.9 M 2023-01-25T11:37:57.277 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 17 M 2023-01-25T11:37:57.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T11:37:57.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T11:37:57.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2023-01-25T11:37:57.277 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T11:37:57.278 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T11:37:57.278 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2023-01-25T11:37:57.278 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T11:37:57.278 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T11:37:57.278 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T11:37:57.278 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2023-01-25T11:37:57.279 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2023-01-25T11:37:57.279 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T11:37:57.279 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T11:37:57.279 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T11:37:57.279 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2023-01-25T11:37:57.280 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2023-01-25T11:37:57.280 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2023-01-25T11:37:57.280 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T11:37:57.280 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T11:37:57.280 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T11:37:57.281 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T11:37:57.281 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T11:37:57.281 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T11:37:57.281 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T11:37:57.281 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2023-01-25T11:37:57.282 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T11:37:57.282 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:57.282 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:37:57.282 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================= 2023-01-25T11:37:57.282 INFO:teuthology.orchestra.run.smithi119.stdout:Install 32 Packages 2023-01-25T11:37:57.283 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:37:57.283 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 28 M 2023-01-25T11:37:57.283 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 107 M 2023-01-25T11:37:57.284 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:37:57.333 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T11:37:57.651 INFO:teuthology.orchestra.run.smithi119.stdout:(1/32): ceph-18.0.0-1986.g44e4095d.el8.x86_64.r 17 kB/s | 6.4 kB 00:00 2023-01-25T11:37:57.730 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T11:37:58.033 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T11:37:58.193 INFO:teuthology.orchestra.run.smithi119.stdout:(2/32): ceph-mds-18.0.0-1986.g44e4095d.el8.x86_ 2.5 MB/s | 2.3 MB 00:00 2023-01-25T11:37:58.221 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T11:37:58.226 INFO:teuthology.orchestra.run.smithi119.stdout:(3/32): ceph-mgr-18.0.0-1986.g44e4095d.el8.x86_ 1.6 MB/s | 1.5 MB 00:00 2023-01-25T11:37:58.447 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T11:37:58.668 INFO:teuthology.orchestra.run.smithi119.stdout:(4/32): ceph-mon-18.0.0-1986.g44e4095d.el8.x86_ 3.8 MB/s | 3.9 MB 00:01 2023-01-25T11:37:58.672 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T11:37:58.727 INFO:teuthology.orchestra.run.smithi119.stdout:(5/32): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-01-25T11:37:58.779 INFO:teuthology.orchestra.run.smithi119.stdout:(6/32): python3-markupsafe-0.23-19.el8.x86_64.r 753 kB/s | 39 kB 00:00 2023-01-25T11:37:58.811 INFO:teuthology.orchestra.run.smithi119.stdout:(7/32): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 3.2 MB/s | 103 kB 00:00 2023-01-25T11:37:58.836 INFO:teuthology.orchestra.run.smithi119.stdout:(8/32): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2023-01-25T11:37:58.870 INFO:teuthology.orchestra.run.smithi119.stdout:(9/32): python3-werkzeug-0.12.2-4.el8.noarch.rp 13 MB/s | 457 kB 00:00 2023-01-25T11:37:58.895 INFO:teuthology.orchestra.run.smithi119.stdout:(10/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 9.3 MB/s | 237 kB 00:00 2023-01-25T11:37:58.929 INFO:teuthology.orchestra.run.smithi119.stdout:(11/32): python3-cryptography-3.2.1-5.el8.x86_6 16 MB/s | 559 kB 00:00 2023-01-25T11:37:58.954 INFO:teuthology.orchestra.run.smithi119.stdout:(12/32): python3-pycparser-2.14-14.el8.noarch.r 4.3 MB/s | 109 kB 00:00 2023-01-25T11:37:59.004 INFO:teuthology.orchestra.run.smithi119.stdout:(13/32): ceph-mgr-modules-core-18.0.0-1986.g44e 303 kB/s | 236 kB 00:00 2023-01-25T11:37:59.005 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T11:37:59.219 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T11:37:59.405 INFO:teuthology.orchestra.run.smithi119.stdout:(14/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 97 kB/s | 44 kB 00:00 2023-01-25T11:37:59.447 INFO:teuthology.orchestra.run.smithi119.stdout:(15/32): python3-beautifulsoup4-4.6.3-2.el8.1.n 419 kB/s | 185 kB 00:00 2023-01-25T11:37:59.474 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T11:37:59.530 INFO:teuthology.orchestra.run.smithi119.stdout:(16/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.3 MB/s | 173 kB 00:00 2023-01-25T11:37:59.581 INFO:teuthology.orchestra.run.smithi119.stdout:(17/32): python3-jaraco-6.2-6.el8.noarch.rpm 222 kB/s | 11 kB 00:00 2023-01-25T11:37:59.631 INFO:teuthology.orchestra.run.smithi119.stdout:(18/32): python3-jaraco-functools-2.0-4.el8.noa 351 kB/s | 18 kB 00:00 2023-01-25T11:37:59.656 INFO:teuthology.orchestra.run.smithi119.stdout:(19/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.8 MB/s | 384 kB 00:00 2023-01-25T11:37:59.676 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T11:37:59.707 INFO:teuthology.orchestra.run.smithi119.stdout:(20/32): python3-logutils-0.3.5-11.el8.noarch.r 646 kB/s | 49 kB 00:00 2023-01-25T11:37:59.732 INFO:teuthology.orchestra.run.smithi119.stdout:(21/32): python3-more-itertools-7.2.0-3.el8.noa 779 kB/s | 59 kB 00:00 2023-01-25T11:37:59.921 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T11:37:59.941 INFO:teuthology.orchestra.run.smithi119.stdout:(22/32): ceph-osd-18.0.0-1986.g44e4095d.el8.x86 9.5 MB/s | 17 MB 00:01 2023-01-25T11:37:59.966 INFO:teuthology.orchestra.run.smithi119.stdout:(23/32): python3-portend-2.6-1.el8.noarch.rpm 67 kB/s | 16 kB 00:00 2023-01-25T11:38:00.024 INFO:teuthology.orchestra.run.smithi119.stdout:(24/32): python3-pecan-1.3.2-9.el8.noarch.rpm 891 kB/s | 283 kB 00:00 2023-01-25T11:38:00.050 INFO:teuthology.orchestra.run.smithi119.stdout:(25/32): python3-singledispatch-3.4.0.3-18.el8. 286 kB/s | 24 kB 00:00 2023-01-25T11:38:00.075 INFO:teuthology.orchestra.run.smithi119.stdout:(26/32): python3-tempora-1.14.1-5.el8.noarch.rp 582 kB/s | 29 kB 00:00 2023-01-25T11:38:00.100 INFO:teuthology.orchestra.run.smithi119.stdout:(27/32): python3-trustme-0.6.0-4.el8.noarch.rpm 546 kB/s | 27 kB 00:00 2023-01-25T11:38:00.125 INFO:teuthology.orchestra.run.smithi119.stdout:(28/32): python3-simplegeneric-0.8.1-17.el8.noa 101 kB/s | 19 kB 00:00 2023-01-25T11:38:00.166 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T11:38:00.226 INFO:teuthology.orchestra.run.smithi119.stdout:(29/32): python3-webtest-2.0.33-1.el8.noarch.rp 854 kB/s | 85 kB 00:00 2023-01-25T11:38:00.251 INFO:teuthology.orchestra.run.smithi119.stdout:(30/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.3 MB/s | 241 kB 00:00 2023-01-25T11:38:00.314 INFO:teuthology.orchestra.run.smithi119.stdout:(31/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.1 MB/s | 251 kB 00:00 2023-01-25T11:38:00.343 INFO:teuthology.orchestra.run.smithi119.stdout:(32/32): python3-zc-lockfile-2.0-2.el8.noarch.r 196 kB/s | 23 kB 00:00 2023-01-25T11:38:00.346 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:00.346 INFO:teuthology.orchestra.run.smithi119.stdout:Total 9.1 MB/s | 28 MB 00:03 2023-01-25T11:38:00.407 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T11:38:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:38:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:38:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:38:00.688 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:38:00.691 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:38:00.767 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T11:38:01.101 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T11:38:01.453 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 27/32 2023-01-25T11:38:01.670 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T11:38:01.972 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:38:02.258 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T11:38:02.562 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T11:38:02.841 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 3/32 2023-01-25T11:38:03.110 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 4/32 2023-01-25T11:38:03.120 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T11:38:03.299 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 5/32 2023-01-25T11:38:03.344 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T11:38:03.530 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 6/32 2023-01-25T11:38:03.730 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T11:38:03.756 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 7/32 2023-01-25T11:38:03.930 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T11:38:03.942 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 8/32 2023-01-25T11:38:04.144 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T11:38:04.193 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 9/32 2023-01-25T11:38:04.321 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T11:38:04.339 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T11:38:04.452 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T11:38:04.481 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 10/32 2023-01-25T11:38:04.760 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 11/32 2023-01-25T11:38:04.902 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T11:38:04.903 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 1/32 2023-01-25T11:38:04.903 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2/32 2023-01-25T11:38:04.903 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 3/32 2023-01-25T11:38:04.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 4/32 2023-01-25T11:38:04.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 5/32 2023-01-25T11:38:04.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 6/32 2023-01-25T11:38:04.904 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T11:38:04.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T11:38:04.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T11:38:04.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T11:38:04.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T11:38:04.905 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T11:38:04.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T11:38:04.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T11:38:04.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T11:38:04.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T11:38:04.906 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T11:38:04.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T11:38:04.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T11:38:04.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T11:38:04.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T11:38:04.907 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T11:38:04.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T11:38:04.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T11:38:04.908 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T11:38:04.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T11:38:04.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T11:38:04.909 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T11:38:04.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T11:38:04.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T11:38:04.910 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T11:38:04.975 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 12/32 2023-01-25T11:38:05.381 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 13/32 2023-01-25T11:38:05.689 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 14/32 2023-01-25T11:38:05.889 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T11:38:05.889 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:05.889 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:05.890 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:05.890 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:05.890 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:05.890 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:05.891 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:05.891 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:05.891 INFO:teuthology.orchestra.run.smithi157.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T11:38:05.891 INFO:teuthology.orchestra.run.smithi157.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T11:38:05.891 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T11:38:05.892 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T11:38:05.892 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T11:38:05.892 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T11:38:05.892 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T11:38:05.893 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T11:38:05.893 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T11:38:05.893 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T11:38:05.893 INFO:teuthology.orchestra.run.smithi157.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T11:38:05.894 INFO:teuthology.orchestra.run.smithi157.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T11:38:05.894 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T11:38:05.894 INFO:teuthology.orchestra.run.smithi157.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T11:38:05.894 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T11:38:05.894 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T11:38:05.895 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T11:38:05.895 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T11:38:05.895 INFO:teuthology.orchestra.run.smithi157.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T11:38:05.895 INFO:teuthology.orchestra.run.smithi157.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T11:38:05.896 INFO:teuthology.orchestra.run.smithi157.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T11:38:05.896 INFO:teuthology.orchestra.run.smithi157.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T11:38:05.896 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T11:38:05.896 INFO:teuthology.orchestra.run.smithi157.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T11:38:05.897 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T11:38:05.897 INFO:teuthology.orchestra.run.smithi157.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T11:38:05.897 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:05.897 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:05.910 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 15/32 2023-01-25T11:38:06.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-base 2023-01-25T11:38:06.127 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 16/32 2023-01-25T11:38:06.345 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T11:38:06.695 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 18/32 2023-01-25T11:38:06.836 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:10 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:06.959 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/32 2023-01-25T11:38:07.213 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 20/32 2023-01-25T11:38:07.424 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T11:38:07.691 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T11:38:07.889 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:38:07.939 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T11:38:07.990 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:07.991 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:38:07.991 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:08.071 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install cephadm 2023-01-25T11:38:08.223 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/32 2023-01-25T11:38:08.597 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 25/32 2023-01-25T11:38:08.787 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:12 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:08.973 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T11:38:09.382 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 27/32 2023-01-25T11:38:09.612 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T11:38:09.924 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:09.925 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:09.925 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:38:09.925 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:09.925 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:09.926 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 82 k 2023-01-25T11:38:09.926 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:09.926 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:09.926 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:09.926 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:38:09.926 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:09.927 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 82 k 2023-01-25T11:38:09.927 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 374 k 2023-01-25T11:38:09.927 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:10.127 INFO:teuthology.orchestra.run.smithi157.stdout:cephadm-18.0.0-1986.g44e4095d.el8.noarch.rpm 410 kB/s | 82 kB 00:00 2023-01-25T11:38:10.128 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:10.128 INFO:teuthology.orchestra.run.smithi157.stdout:Total 408 kB/s | 82 kB 00:00 2023-01-25T11:38:10.128 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:10.135 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:10.135 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:10.137 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:10.141 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:10.188 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:11.062 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 28/32 2023-01-25T11:38:11.315 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T11:38:11.700 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 29/32 2023-01-25T11:38:11.819 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:11.944 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T11:38:11.975 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:12.158 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 30/32 2023-01-25T11:38:12.225 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T11:38:12.387 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 31/32 2023-01-25T11:38:12.492 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T11:38:12.707 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:12.707 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:12.707 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:12.707 INFO:teuthology.orchestra.run.smithi157.stdout: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:12.708 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:12.708 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:12.953 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 32/32 2023-01-25T11:38:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 1/32 2023-01-25T11:38:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2/32 2023-01-25T11:38:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 3/32 2023-01-25T11:38:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 4/32 2023-01-25T11:38:12.954 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 5/32 2023-01-25T11:38:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8. 6/32 2023-01-25T11:38:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 7/32 2023-01-25T11:38:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 8/32 2023-01-25T11:38:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 9/32 2023-01-25T11:38:12.955 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 10/32 2023-01-25T11:38:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 11/32 2023-01-25T11:38:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 12/32 2023-01-25T11:38:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 13/32 2023-01-25T11:38:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 14/32 2023-01-25T11:38:12.956 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 15/32 2023-01-25T11:38:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 16/32 2023-01-25T11:38:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 17/32 2023-01-25T11:38:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 18/32 2023-01-25T11:38:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 19/32 2023-01-25T11:38:12.957 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 20/32 2023-01-25T11:38:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 21/32 2023-01-25T11:38:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 22/32 2023-01-25T11:38:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 23/32 2023-01-25T11:38:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-2.6-1.el8.noarch 24/32 2023-01-25T11:38:12.958 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 25/32 2023-01-25T11:38:12.959 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 26/32 2023-01-25T11:38:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 27/32 2023-01-25T11:38:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 28/32 2023-01-25T11:38:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 29/32 2023-01-25T11:38:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 30/32 2023-01-25T11:38:12.960 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 31/32 2023-01-25T11:38:12.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T11:38:13.691 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:17 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:13.877 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 32/32 2023-01-25T11:38:13.877 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:13.877 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:38:13.877 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:13.878 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:13.878 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:13.878 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:13.878 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:13.878 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:13.879 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T11:38:13.879 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T11:38:13.879 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T11:38:13.879 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T11:38:13.879 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T11:38:13.880 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T11:38:13.880 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T11:38:13.880 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T11:38:13.880 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T11:38:13.880 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T11:38:13.881 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T11:38:13.881 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T11:38:13.881 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T11:38:13.881 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T11:38:13.881 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T11:38:13.882 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T11:38:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T11:38:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T11:38:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T11:38:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T11:38:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T11:38:13.883 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:13.884 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:14.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-base 2023-01-25T11:38:14.817 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:18 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:14.839 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:14.840 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:14.840 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2023-01-25T11:38:14.840 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:14.840 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:14.841 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 165 k 2023-01-25T11:38:14.841 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:14.841 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:14.841 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:14.841 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:38:14.842 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:14.843 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 165 k 2023-01-25T11:38:14.843 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 503 k 2023-01-25T11:38:14.844 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:15.242 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-immutable-object-cache-18.0.0-1986.g44e409 413 kB/s | 165 kB 00:00 2023-01-25T11:38:15.242 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:15.243 INFO:teuthology.orchestra.run.smithi157.stdout:Total 411 kB/s | 165 kB 00:00 2023-01-25T11:38:15.243 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:15.254 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:15.254 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:15.305 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:15.308 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:15.437 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:15.569 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T11:38:15.852 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-base-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:38:15.951 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:15.953 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:38:15.953 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:16.007 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T11:38:16.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install cephadm 2023-01-25T11:38:16.467 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T11:38:16.467 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:16.467 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:16.468 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:16.468 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:16.468 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:16.690 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr 2023-01-25T11:38:16.767 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:20 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:17.430 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:21 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:17.900 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:17.901 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:17.901 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:38:17.901 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:17.901 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:38:17.901 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 82 k 2023-01-25T11:38:17.902 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:17.902 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:38:17.902 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:17.902 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:38:17.902 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:17.903 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 82 k 2023-01-25T11:38:17.903 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 374 k 2023-01-25T11:38:17.903 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:38:18.085 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm-18.0.0-1986.g44e4095d.el8.noarch.rpm 449 kB/s | 82 kB 00:00 2023-01-25T11:38:18.085 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:18.086 INFO:teuthology.orchestra.run.smithi119.stdout:Total 447 kB/s | 82 kB 00:00 2023-01-25T11:38:18.086 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:38:18.093 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:38:18.093 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:38:18.095 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:38:18.099 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:38:18.142 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:38:18.465 INFO:teuthology.orchestra.run.smithi157.stdout:Package ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:38:18.565 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:18.566 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:38:18.567 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:18.655 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T11:38:19.382 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:23 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:19.783 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:20.214 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:20.464 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:20.532 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:20.533 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:20.534 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:38:20.534 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:20.534 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:20.534 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 1.7 M 2023-01-25T11:38:20.534 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:38:20.535 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 23 k 2023-01-25T11:38:20.535 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 15 k 2023-01-25T11:38:20.535 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T11:38:20.535 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T11:38:20.535 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T11:38:20.536 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:20.536 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:20.536 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:20.536 INFO:teuthology.orchestra.run.smithi157.stdout:Install 6 Packages 2023-01-25T11:38:20.536 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:20.537 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 2.0 M 2023-01-25T11:38:20.537 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 8.2 M 2023-01-25T11:38:20.537 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:21.010 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:38:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:21.011 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:21.153 INFO:teuthology.orchestra.run.smithi157.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1986.g44e 37 kB/s | 23 kB 00:00 2023-01-25T11:38:21.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T11:38:21.378 INFO:teuthology.orchestra.run.smithi157.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1986.g44e4 17 kB/s | 15 kB 00:00 2023-01-25T11:38:21.428 INFO:teuthology.orchestra.run.smithi157.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 155 kB/s | 43 kB 00:00 2023-01-25T11:38:21.479 INFO:teuthology.orchestra.run.smithi157.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 336 kB/s | 34 kB 00:00 2023-01-25T11:38:21.538 INFO:teuthology.orchestra.run.smithi157.stdout:(5/6): ceph-mgr-dashboard-18.0.0-1986.g44e4095d 1.7 MB/s | 1.7 MB 00:01 2023-01-25T11:38:21.588 INFO:teuthology.orchestra.run.smithi157.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.2 MB/s | 196 kB 00:00 2023-01-25T11:38:21.590 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:21.591 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.9 MB/s | 2.0 MB 00:01 2023-01-25T11:38:21.602 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:21.612 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:21.612 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:21.699 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:21.703 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:21.939 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:25 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:21.974 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:22.190 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T11:38:22.385 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T11:38:22.553 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T11:38:22.724 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 4/6 2023-01-25T11:38:23.082 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 5/6 2023-01-25T11:38:23.084 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:23.084 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:23.085 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2023-01-25T11:38:23.085 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:23.085 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:38:23.085 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 165 k 2023-01-25T11:38:23.086 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:23.086 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:38:23.086 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:23.086 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:38:23.086 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:23.087 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 165 k 2023-01-25T11:38:23.087 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 503 k 2023-01-25T11:38:23.088 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:38:23.290 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-immutable-object-cache-18.0.0-1986.g44e409 810 kB/s | 165 kB 00:00 2023-01-25T11:38:23.291 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:23.292 INFO:teuthology.orchestra.run.smithi119.stdout:Total 804 kB/s | 165 kB 00:00 2023-01-25T11:38:23.292 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:38:23.299 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T11:38:23.302 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:38:23.303 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:38:23.353 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:38:23.356 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:38:23.484 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:38:23.609 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T11:38:23.609 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 1/6 2023-01-25T11:38:23.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 2/6 2023-01-25T11:38:23.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 3/6 2023-01-25T11:38:23.610 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T11:38:23.611 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T11:38:23.659 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T11:38:24.105 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T11:38:24.105 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:24.105 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:24.105 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:24.106 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:24.106 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:24.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T11:38:24.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T11:38:24.106 INFO:teuthology.orchestra.run.smithi157.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T11:38:24.107 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:24.107 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:24.116 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T11:38:24.207 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T11:38:24.617 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d. 1/1 2023-01-25T11:38:24.617 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:24.618 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:38:24.618 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:38:24.618 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:24.618 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:24.845 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr 2023-01-25T11:38:24.932 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:28 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:25.579 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:29 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:26.086 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:26.088 INFO:teuthology.orchestra.run.smithi157.stdout:===================================================================================================== 2023-01-25T11:38:26.088 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:38:26.088 INFO:teuthology.orchestra.run.smithi157.stdout:===================================================================================================== 2023-01-25T11:38:26.088 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 7.4 M 2023-01-25T11:38:26.089 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:38:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T11:38:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T11:38:26.089 INFO:teuthology.orchestra.run.smithi157.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T11:38:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T11:38:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T11:38:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T11:38:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T11:38:26.090 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:26.091 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:26.091 INFO:teuthology.orchestra.run.smithi157.stdout:===================================================================================================== 2023-01-25T11:38:26.091 INFO:teuthology.orchestra.run.smithi157.stdout:Install 8 Packages 2023-01-25T11:38:26.091 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:26.092 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 35 M 2023-01-25T11:38:26.092 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 210 M 2023-01-25T11:38:26.092 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:26.269 INFO:teuthology.orchestra.run.smithi157.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 27 MB/s | 4.7 MB 00:00 2023-01-25T11:38:26.419 INFO:teuthology.orchestra.run.smithi157.stdout:(2/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2023-01-25T11:38:26.544 INFO:teuthology.orchestra.run.smithi157.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.8 MB/s | 225 kB 00:00 2023-01-25T11:38:26.603 INFO:teuthology.orchestra.run.smithi157.stdout:(4/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 11 MB/s | 3.7 MB 00:00 2023-01-25T11:38:26.613 INFO:teuthology.orchestra.run.smithi119.stdout:Package ceph-mgr-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:38:26.654 INFO:teuthology.orchestra.run.smithi157.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 13 MB/s | 645 kB 00:00 2023-01-25T11:38:26.687 INFO:teuthology.orchestra.run.smithi157.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 5.1 MB/s | 172 kB 00:00 2023-01-25T11:38:26.713 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:26.714 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:38:26.714 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:26.779 INFO:teuthology.orchestra.run.smithi157.stdout:(7/8): ceph-mgr-diskprediction-local-18.0.0-198 11 MB/s | 7.4 MB 00:00 2023-01-25T11:38:26.996 INFO:teuthology.orchestra.run.smithi157.stdout:(8/8): python3-scipy-1.0.0-21.module_el8.5.0+77 30 MB/s | 14 MB 00:00 2023-01-25T11:38:26.999 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:26.999 INFO:teuthology.orchestra.run.smithi157.stdout:Total 39 MB/s | 35 MB 00:00 2023-01-25T11:38:27.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T11:38:27.229 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:27.247 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:27.248 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:27.567 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:27.571 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:27.868 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:31 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:28.140 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:28.275 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T11:38:28.395 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T11:38:28.525 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T11:38:29.022 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:29.023 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:29.023 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:38:29.023 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:29.023 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:38:29.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 1.7 M 2023-01-25T11:38:29.024 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:38:29.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 23 k 2023-01-25T11:38:29.024 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 15 k 2023-01-25T11:38:29.024 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2023-01-25T11:38:29.025 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T11:38:29.025 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T11:38:29.025 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:29.025 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:38:29.025 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:29.026 INFO:teuthology.orchestra.run.smithi119.stdout:Install 6 Packages 2023-01-25T11:38:29.026 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:29.027 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 2.0 M 2023-01-25T11:38:29.028 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 8.2 M 2023-01-25T11:38:29.028 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:38:29.121 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T11:38:29.229 INFO:teuthology.orchestra.run.smithi119.stdout:(1/6): ceph-prometheus-alerts-18.0.0-1986.g44e4 72 kB/s | 15 kB 00:00 2023-01-25T11:38:29.261 INFO:teuthology.orchestra.run.smithi119.stdout:(2/6): ceph-grafana-dashboards-18.0.0-1986.g44e 98 kB/s | 23 kB 00:00 2023-01-25T11:38:29.291 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T11:38:29.312 INFO:teuthology.orchestra.run.smithi119.stdout:(3/6): python3-jwt-1.6.1-2.el8.noarch.rpm 517 kB/s | 43 kB 00:00 2023-01-25T11:38:29.370 INFO:teuthology.orchestra.run.smithi119.stdout:(4/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 309 kB/s | 34 kB 00:00 2023-01-25T11:38:29.412 INFO:teuthology.orchestra.run.smithi119.stdout:(5/6): ceph-mgr-dashboard-18.0.0-1986.g44e4095d 4.5 MB/s | 1.7 MB 00:00 2023-01-25T11:38:29.471 INFO:teuthology.orchestra.run.smithi119.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.2 MB/s | 196 kB 00:00 2023-01-25T11:38:29.473 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:29.473 INFO:teuthology.orchestra.run.smithi119.stdout:Total 4.5 MB/s | 2.0 MB 00:00 2023-01-25T11:38:29.483 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:38:29.492 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:38:29.492 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:38:29.573 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:38:29.577 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:38:29.858 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:38:29.906 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T11:38:30.079 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2023-01-25T11:38:30.099 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T11:38:30.324 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2023-01-25T11:38:30.543 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2023-01-25T11:38:30.689 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 4/6 2023-01-25T11:38:30.782 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T11:38:31.094 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 5/6 2023-01-25T11:38:31.112 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T11:38:31.303 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T11:38:31.341 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T11:38:31.632 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 6/6 2023-01-25T11:38:31.632 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8. 1/6 2023-01-25T11:38:31.632 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarc 2/6 2023-01-25T11:38:31.632 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.n 3/6 2023-01-25T11:38:31.633 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2023-01-25T11:38:31.633 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2023-01-25T11:38:32.124 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2023-01-25T11:38:32.125 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:32.125 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:38:32.125 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:32.126 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:32.126 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:32.126 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T11:38:32.127 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T11:38:32.127 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T11:38:32.127 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:32.128 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:32.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T11:38:32.964 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:36 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:33.087 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T11:38:34.098 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:34.099 INFO:teuthology.orchestra.run.smithi119.stdout:===================================================================================================== 2023-01-25T11:38:34.099 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:38:34.099 INFO:teuthology.orchestra.run.smithi119.stdout:===================================================================================================== 2023-01-25T11:38:34.099 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:38:34.100 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 7.4 M 2023-01-25T11:38:34.100 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:38:34.100 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 8.5.0-18.el8 CentOS-BaseOS 645 k 2023-01-25T11:38:34.100 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 8.5.0-18.el8 CentOS-BaseOS 172 k 2023-01-25T11:38:34.100 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2023-01-25T11:38:34.101 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2023-01-25T11:38:34.101 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2023-01-25T11:38:34.101 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2023-01-25T11:38:34.101 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2023-01-25T11:38:34.101 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:34.102 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:38:34.102 INFO:teuthology.orchestra.run.smithi119.stdout:===================================================================================================== 2023-01-25T11:38:34.102 INFO:teuthology.orchestra.run.smithi119.stdout:Install 8 Packages 2023-01-25T11:38:34.102 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:34.103 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 35 M 2023-01-25T11:38:34.103 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 210 M 2023-01-25T11:38:34.103 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:38:34.260 INFO:teuthology.orchestra.run.smithi119.stdout:(1/8): openblas-0.3.15-4.el8.x86_64.rpm 30 MB/s | 4.7 MB 00:00 2023-01-25T11:38:34.346 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T11:38:34.352 INFO:teuthology.orchestra.run.smithi119.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 41 MB/s | 3.7 MB 00:00 2023-01-25T11:38:34.385 INFO:teuthology.orchestra.run.smithi119.stdout:(3/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 7.0 MB/s | 225 kB 00:00 2023-01-25T11:38:34.507 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T11:38:34.669 INFO:teuthology.orchestra.run.smithi119.stdout:(4/8): python3-scipy-1.0.0-21.module_el8.5.0+77 48 MB/s | 14 MB 00:00 2023-01-25T11:38:34.703 INFO:teuthology.orchestra.run.smithi119.stdout:(5/8): libgfortran-8.5.0-18.el8.x86_64.rpm 19 MB/s | 645 kB 00:00 2023-01-25T11:38:34.728 INFO:teuthology.orchestra.run.smithi119.stdout:(6/8): libquadmath-8.5.0-18.el8.x86_64.rpm 6.7 MB/s | 172 kB 00:00 2023-01-25T11:38:34.837 INFO:teuthology.orchestra.run.smithi119.stdout:(7/8): openblas-threads-0.3.15-4.el8.x86_64.rpm 6.6 MB/s | 4.9 MB 00:00 2023-01-25T11:38:34.845 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T11:38:34.845 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 1/8 2023-01-25T11:38:34.845 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T11:38:34.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T11:38:34.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T11:38:34.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T11:38:34.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T11:38:34.846 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T11:38:35.364 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T11:38:35.364 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:35.364 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:35.364 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:35.365 INFO:teuthology.orchestra.run.smithi157.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T11:38:35.365 INFO:teuthology.orchestra.run.smithi157.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T11:38:35.365 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T11:38:35.365 INFO:teuthology.orchestra.run.smithi157.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T11:38:35.365 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T11:38:35.366 INFO:teuthology.orchestra.run.smithi157.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T11:38:35.366 INFO:teuthology.orchestra.run.smithi157.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T11:38:35.366 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:35.366 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:35.470 INFO:teuthology.orchestra.run.smithi119.stdout:(8/8): ceph-mgr-diskprediction-local-18.0.0-198 5.4 MB/s | 7.4 MB 00:01 2023-01-25T11:38:35.473 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:35.473 INFO:teuthology.orchestra.run.smithi119.stdout:Total 26 MB/s | 35 MB 00:01 2023-01-25T11:38:35.479 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-rook 2023-01-25T11:38:35.703 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:38:35.721 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:38:35.722 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:38:36.041 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:38:36.044 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:38:36.209 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:40 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:36.622 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:38:36.823 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T11:38:36.948 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libquadmath-8.5.0-18.el8.x86_64 1/8 2023-01-25T11:38:37.131 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T11:38:37.346 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:37.348 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2023-01-25T11:38:37.348 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:38:37.349 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2023-01-25T11:38:37.349 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:37.349 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 47 k 2023-01-25T11:38:37.349 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:38:37.349 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T11:38:37.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T11:38:37.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T11:38:37.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T11:38:37.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T11:38:37.350 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T11:38:37.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T11:38:37.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T11:38:37.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T11:38:37.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T11:38:37.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T11:38:37.351 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T11:38:37.352 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:37.352 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:37.352 INFO:teuthology.orchestra.run.smithi157.stdout:======================================================================================== 2023-01-25T11:38:37.352 INFO:teuthology.orchestra.run.smithi157.stdout:Install 13 Packages 2023-01-25T11:38:37.352 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:37.353 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 1.9 M 2023-01-25T11:38:37.353 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 25 M 2023-01-25T11:38:37.353 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:37.663 INFO:teuthology.orchestra.run.smithi157.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 85 kB/s | 27 kB 00:00 2023-01-25T11:38:37.729 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libgfortran-8.5.0-18.el8.x86_64 2/8 2023-01-25T11:38:37.746 INFO:teuthology.orchestra.run.smithi157.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 51 kB/s | 20 kB 00:00 2023-01-25T11:38:37.772 INFO:teuthology.orchestra.run.smithi157.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.1 MB/s | 126 kB 00:00 2023-01-25T11:38:37.797 INFO:teuthology.orchestra.run.smithi157.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-25T11:38:37.822 INFO:teuthology.orchestra.run.smithi157.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 3.0 MB/s | 155 kB 00:00 2023-01-25T11:38:37.848 INFO:teuthology.orchestra.run.smithi157.stdout:(6/13): ceph-mgr-rook-18.0.0-1986.g44e4095d.el8 95 kB/s | 47 kB 00:00 2023-01-25T11:38:37.873 INFO:teuthology.orchestra.run.smithi157.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 564 kB/s | 43 kB 00:00 2023-01-25T11:38:37.923 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T11:38:37.973 INFO:teuthology.orchestra.run.smithi157.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 130 kB/s | 16 kB 00:00 2023-01-25T11:38:37.999 INFO:teuthology.orchestra.run.smithi157.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 187 kB/s | 33 kB 00:00 2023-01-25T11:38:38.024 INFO:teuthology.orchestra.run.smithi157.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 540 kB/s | 81 kB 00:00 2023-01-25T11:38:38.074 INFO:teuthology.orchestra.run.smithi157.stdout:(11/13): python3-websocket-client-0.56.0-5.el8. 1.2 MB/s | 61 kB 00:00 2023-01-25T11:38:38.100 INFO:teuthology.orchestra.run.smithi157.stdout:(12/13): python3-rsa-4.8-1.el8.noarch.rpm 612 kB/s | 61 kB 00:00 2023-01-25T11:38:38.225 INFO:teuthology.orchestra.run.smithi157.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 4.6 MB/s | 1.2 MB 00:00 2023-01-25T11:38:38.227 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:38.227 INFO:teuthology.orchestra.run.smithi157.stdout:Total 2.2 MB/s | 1.9 MB 00:00 2023-01-25T11:38:38.285 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:38.301 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:38.301 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:38.400 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:38.403 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:38.575 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 3/8 2023-01-25T11:38:38.756 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T11:38:38.932 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:39.226 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T11:38:39.441 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T11:38:39.443 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 4/8 2023-01-25T11:38:39.725 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T11:38:39.893 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T11:38:40.005 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T11:38:40.156 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T11:38:40.212 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T11:38:40.418 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T11:38:40.710 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T11:38:40.952 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T11:38:41.891 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T11:38:42.066 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T11:38:42.346 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T11:38:42.546 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T11:38:42.740 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T11:38:43.179 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 7/8 2023-01-25T11:38:43.264 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T11:38:43.583 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T11:38:43.584 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 1/13 2023-01-25T11:38:43.584 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T11:38:43.584 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T11:38:43.584 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T11:38:43.585 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T11:38:43.585 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T11:38:43.585 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T11:38:43.585 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T11:38:43.585 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T11:38:43.586 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T11:38:43.586 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T11:38:43.586 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T11:38:43.772 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T11:38:44.091 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 8/8 2023-01-25T11:38:44.091 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095 1/8 2023-01-25T11:38:44.091 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 2/8 2023-01-25T11:38:44.091 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 3/8 2023-01-25T11:38:44.092 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 4/8 2023-01-25T11:38:44.092 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T11:38:44.092 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x 6/8 2023-01-25T11:38:44.092 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-8.5.0-18.el8.x86_64 7/8 2023-01-25T11:38:44.150 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T11:38:44.150 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:44.150 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T11:38:44.151 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi157.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi157.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T11:38:44.152 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T11:38:44.153 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T11:38:44.153 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T11:38:44.153 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T11:38:44.154 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:44.154 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:44.253 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T11:38:44.639 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-8.5.0-18.el8.x86_64 8/8 2023-01-25T11:38:44.639 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:44.639 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:38:44.639 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:44.640 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-8.5.0-18.el8.x86_64 2023-01-25T11:38:44.640 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-8.5.0-18.el8.x86_64 2023-01-25T11:38:44.640 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.15-4.el8.x86_64 2023-01-25T11:38:44.640 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-01-25T11:38:44.640 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T11:38:44.641 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T11:38:44.641 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-01-25T11:38:44.641 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:44.641 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:44.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-rook 2023-01-25T11:38:44.982 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:48 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:45.497 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:49 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:46.115 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:46.116 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:46.117 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:38:46.117 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:46.117 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:46.117 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 120 k 2023-01-25T11:38:46.118 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:38:46.118 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T11:38:46.118 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T11:38:46.118 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T11:38:46.119 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T11:38:46.119 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:46.119 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:46.119 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:46.120 INFO:teuthology.orchestra.run.smithi157.stdout:Install 5 Packages 2023-01-25T11:38:46.120 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:46.121 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 6.0 M 2023-01-25T11:38:46.121 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 26 M 2023-01-25T11:38:46.121 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:46.208 INFO:teuthology.orchestra.run.smithi157.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 5.9 MB/s | 538 kB 00:00 2023-01-25T11:38:46.301 INFO:teuthology.orchestra.run.smithi157.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 26 MB/s | 4.8 MB 00:00 2023-01-25T11:38:46.559 INFO:teuthology.orchestra.run.smithi157.stdout:(3/5): python3-natsort-7.1.1-2.el8.noarch.rpm 233 kB/s | 60 kB 00:00 2023-01-25T11:38:46.584 INFO:teuthology.orchestra.run.smithi157.stdout:(4/5): ceph-mgr-cephadm-18.0.0-1986.g44e4095d.e 258 kB/s | 120 kB 00:00 2023-01-25T11:38:46.628 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:46.629 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================== 2023-01-25T11:38:46.630 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:38:46.630 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================== 2023-01-25T11:38:46.630 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:38:46.630 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 47 k 2023-01-25T11:38:46.631 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:38:46.631 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T11:38:46.631 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T11:38:46.631 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T11:38:46.631 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2023-01-25T11:38:46.632 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2023-01-25T11:38:46.632 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T11:38:46.632 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2023-01-25T11:38:46.632 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2023-01-25T11:38:46.632 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2023-01-25T11:38:46.633 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2023-01-25T11:38:46.633 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T11:38:46.633 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T11:38:46.633 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:46.633 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:38:46.634 INFO:teuthology.orchestra.run.smithi119.stdout:======================================================================================== 2023-01-25T11:38:46.634 INFO:teuthology.orchestra.run.smithi119.stdout:Install 13 Packages 2023-01-25T11:38:46.634 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:46.635 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 1.9 M 2023-01-25T11:38:46.635 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 25 M 2023-01-25T11:38:46.635 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:38:46.668 INFO:teuthology.orchestra.run.smithi157.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.0 MB/s | 476 kB 00:00 2023-01-25T11:38:46.669 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:46.670 INFO:teuthology.orchestra.run.smithi157.stdout:Total 11 MB/s | 6.0 MB 00:00 2023-01-25T11:38:46.704 INFO:teuthology.orchestra.run.smithi119.stdout:(1/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 373 kB/s | 27 kB 00:00 2023-01-25T11:38:46.720 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:46.730 INFO:teuthology.orchestra.run.smithi119.stdout:(2/13): python3-jsonpointer-1.10-11.el8.noarch. 207 kB/s | 20 kB 00:00 2023-01-25T11:38:46.731 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:46.732 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:46.755 INFO:teuthology.orchestra.run.smithi119.stdout:(3/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-01-25T11:38:46.780 INFO:teuthology.orchestra.run.smithi119.stdout:(4/13): python3-pyasn1-modules-0.3.7-6.el8.noar 2.1 MB/s | 110 kB 00:00 2023-01-25T11:38:46.806 INFO:teuthology.orchestra.run.smithi119.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 1.7 MB/s | 43 kB 00:00 2023-01-25T11:38:46.831 INFO:teuthology.orchestra.run.smithi119.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 2.0 MB/s | 155 kB 00:00 2023-01-25T11:38:46.839 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:46.842 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:46.881 INFO:teuthology.orchestra.run.smithi119.stdout:(7/13): ceph-mgr-rook-18.0.0-1986.g44e4095d.el8 191 kB/s | 47 kB 00:00 2023-01-25T11:38:47.007 INFO:teuthology.orchestra.run.smithi119.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 93 kB/s | 16 kB 00:00 2023-01-25T11:38:47.024 INFO:teuthology.orchestra.run.smithi119.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 152 kB/s | 33 kB 00:00 2023-01-25T11:38:47.116 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:47.124 INFO:teuthology.orchestra.run.smithi119.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 335 kB/s | 81 kB 00:00 2023-01-25T11:38:47.157 INFO:teuthology.orchestra.run.smithi119.stdout:(11/13): python3-rsa-4.8-1.el8.noarch.rpm 460 kB/s | 61 kB 00:00 2023-01-25T11:38:47.301 INFO:teuthology.orchestra.run.smithi119.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 346 kB/s | 61 kB 00:00 2023-01-25T11:38:47.461 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T11:38:47.783 INFO:teuthology.orchestra.run.smithi119.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.5 MB/s | 1.2 MB 00:00 2023-01-25T11:38:47.785 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:47.785 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.7 MB/s | 1.9 MB 00:01 2023-01-25T11:38:47.840 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:38:47.856 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:38:47.856 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:38:47.953 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:38:47.957 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:38:48.398 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T11:38:48.486 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:38:48.818 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T11:38:48.982 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T11:38:49.108 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T11:38:49.230 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2023-01-25T11:38:49.288 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T11:38:49.497 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.8-1.el8.noarch 3/13 2023-01-25T11:38:49.591 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T11:38:49.592 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/5 2023-01-25T11:38:49.592 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T11:38:49.592 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T11:38:49.592 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T11:38:49.802 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2023-01-25T11:38:50.051 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2023-01-25T11:38:50.099 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T11:38:50.099 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:50.099 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:50.099 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:50.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T11:38:50.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T11:38:50.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T11:38:50.100 INFO:teuthology.orchestra.run.smithi157.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T11:38:50.100 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:50.101 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:50.210 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-fuse 2023-01-25T11:38:50.315 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2023-01-25T11:38:50.641 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2023-01-25T11:38:50.916 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T11:38:50.949 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:54 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:52.009 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T11:38:52.105 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:52.105 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:52.105 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:38:52.106 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:52.110 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:52.110 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 845 k 2023-01-25T11:38:52.110 INFO:teuthology.orchestra.run.smithi157.stdout:Installing dependencies: 2023-01-25T11:38:52.110 INFO:teuthology.orchestra.run.smithi157.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T11:38:52.110 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:52.111 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:52.111 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:52.111 INFO:teuthology.orchestra.run.smithi157.stdout:Install 2 Packages 2023-01-25T11:38:52.111 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:52.112 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 928 k 2023-01-25T11:38:52.112 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 2.9 M 2023-01-25T11:38:52.112 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:52.184 INFO:teuthology.orchestra.run.smithi157.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2023-01-25T11:38:52.377 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2023-01-25T11:38:52.401 INFO:teuthology.orchestra.run.smithi157.stdout:(2/2): ceph-fuse-18.0.0-1986.g44e4095d.el8.x86_ 2.8 MB/s | 845 kB 00:00 2023-01-25T11:38:52.402 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:52.402 INFO:teuthology.orchestra.run.smithi157.stdout:Total 3.1 MB/s | 928 kB 00:00 2023-01-25T11:38:52.412 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:52.433 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:52.434 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:52.528 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:52.531 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:52.577 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2023-01-25T11:38:52.710 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:52.804 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2023-01-25T11:38:52.945 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T11:38:52.952 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T11:38:53.144 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T11:38:53.260 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 13/13 2023-01-25T11:38:53.261 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 1/13 2023-01-25T11:38:53.261 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2023-01-25T11:38:53.261 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2023-01-25T11:38:53.261 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2023-01-25T11:38:53.262 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2023-01-25T11:38:53.262 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T11:38:53.262 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T11:38:53.262 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2023-01-25T11:38:53.262 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T11:38:53.263 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2023-01-25T11:38:53.263 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2023-01-25T11:38:53.263 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 12/13 2023-01-25T11:38:53.558 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T11:38:53.559 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/2 2023-01-25T11:38:53.853 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2023-01-25T11:38:53.853 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:53.853 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:38:53.854 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:53.854 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T11:38:53.854 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T11:38:53.854 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T11:38:53.855 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T11:38:53.855 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T11:38:53.855 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T11:38:53.855 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T11:38:53.855 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T11:38:53.856 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T11:38:53.856 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T11:38:53.856 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T11:38:53.856 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T11:38:53.856 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:53.857 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:54.007 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T11:38:54.115 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T11:38:54.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:54.116 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:54.116 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T11:38:54.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:54.116 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:54.304 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install ceph-volume 2023-01-25T11:38:54.731 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:58 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:55.043 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:01:59 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:55.865 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:38:55.866 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:55.866 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:38:55.867 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:55.867 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:38:55.867 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 120 k 2023-01-25T11:38:55.867 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:38:55.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T11:38:55.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2023-01-25T11:38:55.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2023-01-25T11:38:55.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T11:38:55.868 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:55.869 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:38:55.869 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:38:55.869 INFO:teuthology.orchestra.run.smithi119.stdout:Install 5 Packages 2023-01-25T11:38:55.869 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:55.870 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 6.0 M 2023-01-25T11:38:55.870 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 26 M 2023-01-25T11:38:55.870 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:38:55.976 INFO:teuthology.orchestra.run.smithi119.stdout:(1/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 4.9 MB/s | 538 kB 00:00 2023-01-25T11:38:56.152 INFO:teuthology.orchestra.run.smithi119.stdout:(2/5): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-01-25T11:38:56.179 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:56.180 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:56.180 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repository Size 2023-01-25T11:38:56.180 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:56.181 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:56.181 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 263 k 2023-01-25T11:38:56.181 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:56.181 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:56.181 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:56.181 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:38:56.182 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:56.182 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 263 k 2023-01-25T11:38:56.183 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 1.3 M 2023-01-25T11:38:56.183 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:56.185 INFO:teuthology.orchestra.run.smithi119.stdout:(3/5): ceph-mgr-cephadm-18.0.0-1986.g44e4095d.e 379 kB/s | 120 kB 00:00 2023-01-25T11:38:56.268 INFO:teuthology.orchestra.run.smithi119.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.6 MB/s | 476 kB 00:00 2023-01-25T11:38:56.285 INFO:teuthology.orchestra.run.smithi119.stdout:(5/5): python3-natsort-7.1.1-2.el8.noarch.rpm 454 kB/s | 60 kB 00:00 2023-01-25T11:38:56.287 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:56.287 INFO:teuthology.orchestra.run.smithi119.stdout:Total 14 MB/s | 6.0 MB 00:00 2023-01-25T11:38:56.338 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:38:56.349 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:38:56.349 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:38:56.398 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-volume-18.0.0-1986.g44e4095d.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-01-25T11:38:56.398 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi157.stdout:Total 1.2 MB/s | 263 kB 00:00 2023-01-25T11:38:56.399 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:38:56.409 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:38:56.409 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:38:56.441 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:38:56.444 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:38:56.454 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:38:56.458 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:38:56.627 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:38:56.745 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:38:56.871 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:57.077 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 1/5 2023-01-25T11:38:57.305 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:57.762 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:38:57.762 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:57.762 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:38:57.762 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:57.763 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:57.763 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:38:57.951 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados-devel 2023-01-25T11:38:58.023 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T11:38:58.440 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 3/5 2023-01-25T11:38:58.689 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:02 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:38:58.739 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T11:38:58.986 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T11:38:59.297 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 5/5 2023-01-25T11:38:59.297 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 1/5 2023-01-25T11:38:59.298 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 2/5 2023-01-25T11:38:59.298 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 3/5 2023-01-25T11:38:59.298 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T11:38:59.795 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 5/5 2023-01-25T11:38:59.796 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:59.796 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:38:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:38:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T11:38:59.797 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T11:38:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T11:38:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T11:38:59.798 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:38:59.799 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:38:59.824 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:38:59.826 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:59.826 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2023-01-25T11:38:59.826 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:59.826 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:38:59.827 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 139 k 2023-01-25T11:38:59.827 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:59.827 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:38:59.827 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:38:59.828 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:38:59.828 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:38:59.828 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 139 k 2023-01-25T11:38:59.828 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 512 k 2023-01-25T11:38:59.828 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:38:59.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-fuse 2023-01-25T11:39:00.178 INFO:teuthology.orchestra.run.smithi157.stdout:librados-devel-18.0.0-1986.g44e4095d.el8.x86_64 396 kB/s | 139 kB 00:00 2023-01-25T11:39:00.178 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:00.179 INFO:teuthology.orchestra.run.smithi157.stdout:Total 395 kB/s | 139 kB 00:00 2023-01-25T11:39:00.179 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:39:00.184 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:39:00.184 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:39:00.238 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:39:00.241 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:39:00.361 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:39:00.591 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:00.673 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:04 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:00.909 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:01.372 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:01.372 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:01.372 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:39:01.373 INFO:teuthology.orchestra.run.smithi157.stdout: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:01.373 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:01.373 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:01.577 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs2 2023-01-25T11:39:01.813 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:01.814 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:39:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:01.814 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:39:01.814 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 845 k 2023-01-25T11:39:01.815 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-01-25T11:39:01.815 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2023-01-25T11:39:01.815 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:01.815 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:39:01.815 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:01.816 INFO:teuthology.orchestra.run.smithi119.stdout:Install 2 Packages 2023-01-25T11:39:01.816 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 928 k 2023-01-25T11:39:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 2.9 M 2023-01-25T11:39:01.817 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:39:01.901 INFO:teuthology.orchestra.run.smithi119.stdout:(1/2): fuse-2.9.7-16.el8.x86_64.rpm 968 kB/s | 83 kB 00:00 2023-01-25T11:39:02.243 INFO:teuthology.orchestra.run.smithi119.stdout:(2/2): ceph-fuse-18.0.0-1986.g44e4095d.el8.x86_ 1.9 MB/s | 845 kB 00:00 2023-01-25T11:39:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:02.244 INFO:teuthology.orchestra.run.smithi119.stdout:Total 2.1 MB/s | 928 kB 00:00 2023-01-25T11:39:02.254 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:39:02.275 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:39:02.276 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:39:02.315 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:06 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:02.369 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:39:02.373 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:39:02.557 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:39:02.867 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.7-16.el8.x86_64 1/2 2023-01-25T11:39:03.109 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T11:39:03.352 INFO:teuthology.orchestra.run.smithi157.stdout:Package libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:03.454 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:03.455 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:39:03.456 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2/2 2023-01-25T11:39:03.509 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/2 2023-01-25T11:39:03.543 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install libcephfs-devel 2023-01-25T11:39:04.002 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 2/2 2023-01-25T11:39:04.003 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:04.003 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:39:04.003 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 fuse-2.9.7-16.el8.x86_64 2023-01-25T11:39:04.004 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:04.004 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:04.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-volume 2023-01-25T11:39:04.264 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:08 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:04.929 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:08 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:05.411 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:05.412 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:05.412 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2023-01-25T11:39:05.412 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:05.413 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:39:05.413 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 30 k 2023-01-25T11:39:05.413 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:05.413 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:39:05.413 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:05.414 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:39:05.414 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:05.414 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 30 k 2023-01-25T11:39:05.414 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 137 k 2023-01-25T11:39:05.414 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:39:06.068 INFO:teuthology.orchestra.run.smithi157.stdout:libcephfs-devel-18.0.0-1986.g44e4095d.el8.x86_6 46 kB/s | 30 kB 00:00 2023-01-25T11:39:06.068 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:06.069 INFO:teuthology.orchestra.run.smithi157.stdout:Total 46 kB/s | 30 kB 00:00 2023-01-25T11:39:06.069 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:39:06.073 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:39:06.073 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:39:06.074 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:06.075 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:06.075 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-01-25T11:39:06.075 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:06.075 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:39:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:18.0.0-1986.g44e4095d.el8 ceph-noarch 263 k 2023-01-25T11:39:06.076 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:06.076 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:39:06.076 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:06.077 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:39:06.077 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:06.078 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 263 k 2023-01-25T11:39:06.078 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 1.3 M 2023-01-25T11:39:06.078 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:39:06.085 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:39:06.089 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:39:06.155 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:39:06.292 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-volume-18.0.0-1986.g44e4095d.el8.noarch.rp 1.2 MB/s | 263 kB 00:00 2023-01-25T11:39:06.293 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:06.293 INFO:teuthology.orchestra.run.smithi119.stdout:Total 1.2 MB/s | 263 kB 00:00 2023-01-25T11:39:06.293 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:39:06.303 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:39:06.303 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:39:06.333 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:39:06.337 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:39:06.349 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:06.524 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:39:06.630 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:06.835 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:39:07.081 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:07.081 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:07.082 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:39:07.082 INFO:teuthology.orchestra.run.smithi157.stdout: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:07.082 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:07.082 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:07.201 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librados2 2023-01-25T11:39:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:39:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 1/1 2023-01-25T11:39:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:07.742 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:39:07.742 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:18.0.0-1986.g44e4095d.el8.noarch 2023-01-25T11:39:07.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:07.743 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:07.919 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:11 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:07.951 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados-devel 2023-01-25T11:39:08.686 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:12 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:08.966 INFO:teuthology.orchestra.run.smithi157.stdout:Package librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:09.067 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:09.069 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:39:09.069 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:09.154 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install librbd1 2023-01-25T11:39:09.824 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:09.825 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2023-01-25T11:39:09.825 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:09.825 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:39:09.825 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 139 k 2023-01-25T11:39:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:09.826 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:39:09.826 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:09.826 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:39:09.826 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:09.827 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 139 k 2023-01-25T11:39:09.827 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 512 k 2023-01-25T11:39:09.827 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:39:09.874 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:13 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:10.199 INFO:teuthology.orchestra.run.smithi119.stdout:librados-devel-18.0.0-1986.g44e4095d.el8.x86_64 372 kB/s | 139 kB 00:00 2023-01-25T11:39:10.200 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:10.200 INFO:teuthology.orchestra.run.smithi119.stdout:Total 371 kB/s | 139 kB 00:00 2023-01-25T11:39:10.201 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:39:10.206 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:39:10.206 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:39:10.260 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:39:10.263 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:39:10.869 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:39:10.912 INFO:teuthology.orchestra.run.smithi157.stdout:Package librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:11.015 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:11.016 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:39:11.016 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:11.106 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rados 2023-01-25T11:39:11.799 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:11.830 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:15 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:12.108 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:12.569 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:39:12.569 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:12.570 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:12.570 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:12.803 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs2 2023-01-25T11:39:12.869 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:12.971 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:12.972 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:39:12.972 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:13.067 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rgw 2023-01-25T11:39:13.539 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:17 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:13.789 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:17 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:14.579 INFO:teuthology.orchestra.run.smithi119.stdout:Package libcephfs2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:14.680 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:14.681 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:39:14.681 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:14.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install libcephfs-devel 2023-01-25T11:39:14.831 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:14.932 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:14.933 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:39:14.933 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:15.033 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-cephfs 2023-01-25T11:39:15.497 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:19 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:15.754 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:19 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:16.644 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:16.645 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:16.645 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2023-01-25T11:39:16.646 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:16.646 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:39:16.646 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 30 k 2023-01-25T11:39:16.646 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:16.646 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:39:16.647 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:16.647 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:39:16.647 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:16.647 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 30 k 2023-01-25T11:39:16.647 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 137 k 2023-01-25T11:39:16.648 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:39:16.801 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:16.903 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:16.904 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:39:16.904 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:16.990 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install python3-rbd 2023-01-25T11:39:17.054 INFO:teuthology.orchestra.run.smithi119.stdout:libcephfs-devel-18.0.0-1986.g44e4095d.el8.x86_6 75 kB/s | 30 kB 00:00 2023-01-25T11:39:17.055 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:17.055 INFO:teuthology.orchestra.run.smithi119.stdout:Total 74 kB/s | 30 kB 00:00 2023-01-25T11:39:17.056 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:39:17.059 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:39:17.059 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:39:17.069 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:39:17.073 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:39:17.134 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:39:17.716 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:21 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:18.264 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:18.582 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:18.762 INFO:teuthology.orchestra.run.smithi157.stdout:Package python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:18.863 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:18.864 INFO:teuthology.orchestra.run.smithi157.stdout:Nothing to do. 2023-01-25T11:39:18.864 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:18.954 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-fuse 2023-01-25T11:39:19.025 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:19.025 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:19.025 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:39:19.026 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:19.026 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:19.026 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:19.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librados2 2023-01-25T11:39:19.677 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:19.879 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:20.819 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:20.820 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:20.820 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2023-01-25T11:39:20.820 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:20.821 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:39:20.821 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 94 k 2023-01-25T11:39:20.821 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:20.822 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:39:20.822 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:20.822 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:39:20.823 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:20.823 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 94 k 2023-01-25T11:39:20.823 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 280 k 2023-01-25T11:39:20.824 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:39:20.930 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:21.007 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-fuse-18.0.0-1986.g44e4095d.el8.x86_64.rpm 508 kB/s | 94 kB 00:00 2023-01-25T11:39:21.007 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:21.008 INFO:teuthology.orchestra.run.smithi157.stdout:Total 505 kB/s | 94 kB 00:00 2023-01-25T11:39:21.008 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:39:21.018 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:39:21.018 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:39:21.031 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:21.032 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:39:21.032 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:21.069 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:39:21.073 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:39:21.130 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install librbd1 2023-01-25T11:39:21.182 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:39:21.380 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:21.687 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:21.849 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:25 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:22.303 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:22.303 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:39:22.304 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:22.304 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:22.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-mirror 2023-01-25T11:39:22.889 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:22.991 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:22.992 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:39:22.992 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:23.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rados 2023-01-25T11:39:23.218 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:27 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:23.818 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:27 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:24.359 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:24.360 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:24.361 INFO:teuthology.orchestra.run.smithi157.stdout: Package Arch Version Repo Size 2023-01-25T11:39:24.361 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:24.361 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:39:24.361 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.5 M 2023-01-25T11:39:24.362 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:24.362 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:39:24.362 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:24.362 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:39:24.362 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:24.363 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 3.5 M 2023-01-25T11:39:24.363 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 16 M 2023-01-25T11:39:24.363 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:39:24.755 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-mirror-18.0.0-1986.g44e4095d.el8.x86_64.rpm 8.8 MB/s | 3.5 MB 00:00 2023-01-25T11:39:24.756 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:24.756 INFO:teuthology.orchestra.run.smithi157.stdout:Total 8.7 MB/s | 3.5 MB 00:00 2023-01-25T11:39:24.757 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:39:24.768 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:39:24.769 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:39:24.832 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:39:24.835 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:39:24.863 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rados-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:24.964 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:24.965 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:39:24.965 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:25.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rgw 2023-01-25T11:39:25.289 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:39:25.433 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:25.777 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:29 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:25.883 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:26.404 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:26.405 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:26.405 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:39:26.405 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:26.405 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:26.406 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:26.593 DEBUG:teuthology.orchestra.run.smithi157:> sudo yum -y install rbd-nbd 2023-01-25T11:39:26.820 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rgw-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:26.922 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:26.923 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:39:26.923 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:27.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-cephfs 2023-01-25T11:39:27.326 INFO:teuthology.orchestra.run.smithi157.stdout:Last metadata expiration check: 0:02:31 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:27.737 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:31 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:28.474 INFO:teuthology.orchestra.run.smithi157.stdout:Dependencies resolved. 2023-01-25T11:39:28.475 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:28.475 INFO:teuthology.orchestra.run.smithi157.stdout: Package Architecture Version Repository Size 2023-01-25T11:39:28.475 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:28.476 INFO:teuthology.orchestra.run.smithi157.stdout:Installing: 2023-01-25T11:39:28.476 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 180 k 2023-01-25T11:39:28.476 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:28.476 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction Summary 2023-01-25T11:39:28.476 INFO:teuthology.orchestra.run.smithi157.stdout:================================================================================ 2023-01-25T11:39:28.477 INFO:teuthology.orchestra.run.smithi157.stdout:Install 1 Package 2023-01-25T11:39:28.477 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:28.478 INFO:teuthology.orchestra.run.smithi157.stdout:Total download size: 180 k 2023-01-25T11:39:28.478 INFO:teuthology.orchestra.run.smithi157.stdout:Installed size: 568 k 2023-01-25T11:39:28.478 INFO:teuthology.orchestra.run.smithi157.stdout:Downloading Packages: 2023-01-25T11:39:28.780 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-cephfs-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:28.882 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:28.884 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:39:28.884 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:28.934 INFO:teuthology.orchestra.run.smithi157.stdout:rbd-nbd-18.0.0-1986.g44e4095d.el8.x86_64.rpm 392 kB/s | 180 kB 00:00 2023-01-25T11:39:28.935 INFO:teuthology.orchestra.run.smithi157.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:28.935 INFO:teuthology.orchestra.run.smithi157.stdout:Total 391 kB/s | 180 kB 00:00 2023-01-25T11:39:28.936 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction check 2023-01-25T11:39:28.946 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction check succeeded. 2023-01-25T11:39:28.947 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction test 2023-01-25T11:39:28.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install python3-rbd 2023-01-25T11:39:29.000 INFO:teuthology.orchestra.run.smithi157.stdout:Transaction test succeeded. 2023-01-25T11:39:29.003 INFO:teuthology.orchestra.run.smithi157.stdout:Running transaction 2023-01-25T11:39:29.124 INFO:teuthology.orchestra.run.smithi157.stdout: Preparing : 1/1 2023-01-25T11:39:29.271 INFO:teuthology.orchestra.run.smithi157.stdout: Installing : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:29.574 INFO:teuthology.orchestra.run.smithi157.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:29.699 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:33 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:30.052 INFO:teuthology.orchestra.run.smithi157.stdout: Verifying : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:30.052 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:30.052 INFO:teuthology.orchestra.run.smithi157.stdout:Installed: 2023-01-25T11:39:30.053 INFO:teuthology.orchestra.run.smithi157.stdout: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:30.053 INFO:teuthology.orchestra.run.smithi157.stdout: 2023-01-25T11:39:30.053 INFO:teuthology.orchestra.run.smithi157.stdout:Complete! 2023-01-25T11:39:30.233 DEBUG:teuthology.parallel:result is None 2023-01-25T11:39:30.737 INFO:teuthology.orchestra.run.smithi119.stdout:Package python3-rbd-2:18.0.0-1986.g44e4095d.el8.x86_64 is already installed. 2023-01-25T11:39:30.837 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:30.838 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-01-25T11:39:30.839 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:30.932 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-fuse 2023-01-25T11:39:31.651 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:35 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:32.789 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:32.790 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:32.790 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2023-01-25T11:39:32.790 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:32.790 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:39:32.790 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 94 k 2023-01-25T11:39:32.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:32.791 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:39:32.791 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:32.791 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:39:32.791 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:32.792 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 94 k 2023-01-25T11:39:32.792 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 280 k 2023-01-25T11:39:32.792 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:39:33.069 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-fuse-18.0.0-1986.g44e4095d.el8.x86_64.rpm 338 kB/s | 94 kB 00:00 2023-01-25T11:39:33.070 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:33.071 INFO:teuthology.orchestra.run.smithi119.stdout:Total 337 kB/s | 94 kB 00:00 2023-01-25T11:39:33.071 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:39:33.081 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:39:33.081 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:39:33.133 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:39:33.136 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:39:33.252 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:39:33.469 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:33.768 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:34.258 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:34.258 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:34.258 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:39:34.259 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:34.259 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:34.259 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:34.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-mirror 2023-01-25T11:39:35.214 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:39 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:36.353 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:36.354 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:36.354 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repo Size 2023-01-25T11:39:36.354 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:36.354 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:39:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 3.5 M 2023-01-25T11:39:36.355 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:36.355 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:39:36.355 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:36.355 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:39:36.356 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:36.356 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 3.5 M 2023-01-25T11:39:36.356 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 16 M 2023-01-25T11:39:36.356 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:39:37.391 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-mirror-18.0.0-1986.g44e4095d.el8.x86_64.rpm 3.3 MB/s | 3.5 MB 00:01 2023-01-25T11:39:37.391 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:37.392 INFO:teuthology.orchestra.run.smithi119.stdout:Total 3.3 MB/s | 3.5 MB 00:01 2023-01-25T11:39:37.392 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:39:37.404 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:39:37.404 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:39:37.467 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:39:37.470 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:39:37.918 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:39:38.121 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:38.576 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:39.075 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:39.075 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:39.075 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:39:39.075 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:39.076 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:39.076 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:39.262 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install rbd-nbd 2023-01-25T11:39:39.995 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:02:43 ago on Wed 25 Jan 2023 11:36:56 AM UTC. 2023-01-25T11:39:41.142 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-01-25T11:39:41.143 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:41.143 INFO:teuthology.orchestra.run.smithi119.stdout: Package Architecture Version Repository Size 2023-01-25T11:39:41.144 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:41.144 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-01-25T11:39:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:18.0.0-1986.g44e4095d.el8 ceph 180 k 2023-01-25T11:39:41.144 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:41.144 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-01-25T11:39:41.145 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-01-25T11:39:41.145 INFO:teuthology.orchestra.run.smithi119.stdout:Install 1 Package 2023-01-25T11:39:41.145 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:41.145 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 180 k 2023-01-25T11:39:41.145 INFO:teuthology.orchestra.run.smithi119.stdout:Installed size: 568 k 2023-01-25T11:39:41.146 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-01-25T11:39:41.445 INFO:teuthology.orchestra.run.smithi119.stdout:rbd-nbd-18.0.0-1986.g44e4095d.el8.x86_64.rpm 598 kB/s | 180 kB 00:00 2023-01-25T11:39:41.446 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:39:41.446 INFO:teuthology.orchestra.run.smithi119.stdout:Total 595 kB/s | 180 kB 00:00 2023-01-25T11:39:41.446 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-01-25T11:39:41.457 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-01-25T11:39:41.457 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-01-25T11:39:41.510 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-01-25T11:39:41.513 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-01-25T11:39:41.636 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-01-25T11:39:41.835 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:42.133 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:42.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 1/1 2023-01-25T11:39:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:42.600 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-01-25T11:39:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:18.0.0-1986.g44e4095d.el8.x86_64 2023-01-25T11:39:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:39:42.600 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-01-25T11:39:42.850 DEBUG:teuthology.parallel:result is None 2023-01-25T11:39:42.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo lsb_release -is 2023-01-25T11:39:42.888 INFO:teuthology.orchestra.run.smithi119.stdout:CentOSStream 2023-01-25T11:39:42.890 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T11:39:42.890 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:39:43.041 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T11:39:43.063 INFO:teuthology.orchestra.run.smithi119.stdout:18.0.0-1986.g44e4095d.el8 2023-01-25T11:39:43.064 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1986.g44e4095d.el8 2023-01-25T11:39:43.064 INFO:teuthology.task.install:The correct ceph version 18.0.0-1986.g44e4095d is installed. 2023-01-25T11:39:43.065 DEBUG:teuthology.orchestra.run.smithi157:> sudo lsb_release -is 2023-01-25T11:39:43.104 INFO:teuthology.orchestra.run.smithi157.stdout:CentOSStream 2023-01-25T11:39:43.106 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T11:39:43.106 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:39:43.309 DEBUG:teuthology.orchestra.run.smithi157:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T11:39:43.332 INFO:teuthology.orchestra.run.smithi157.stdout:18.0.0-1986.g44e4095d.el8 2023-01-25T11:39:43.333 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1986.g44e4095d.el8 2023-01-25T11:39:43.333 INFO:teuthology.task.install:The correct ceph version 18.0.0-1986.g44e4095d is installed. 2023-01-25T11:39:43.334 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T11:39:43.334 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-01-25T11:39:43.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T11:39:43.362 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-01-25T11:39:43.363 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T11:39:43.401 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T11:39:43.401 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-01-25T11:39:43.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T11:39:43.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T11:39:43.497 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-01-25T11:39:43.497 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T11:39:43.525 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T11:39:43.593 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T11:39:43.593 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-01-25T11:39:43.594 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T11:39:43.621 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T11:39:43.688 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2023-01-25T11:39:43.688 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T11:39:43.715 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T11:39:43.782 INFO:teuthology.run_tasks:Running task cephadm... 2023-01-25T11:39:43.854 INFO:tasks.cephadm:Config: {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c'} 2023-01-25T11:39:43.855 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:39:43.865 INFO:tasks.cephadm:Cluster fsid is f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:39:43.865 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-01-25T11:39:43.866 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.119'} 2023-01-25T11:39:43.866 INFO:tasks.cephadm:First mon is mon.a on smithi119 2023-01-25T11:39:43.866 INFO:tasks.cephadm:First mgr is a 2023-01-25T11:39:43.867 INFO:tasks.cephadm:Normalizing hostnames... 2023-01-25T11:39:43.867 DEBUG:teuthology.orchestra.run.smithi119:> sudo hostname $(hostname -s) 2023-01-25T11:39:43.896 DEBUG:teuthology.orchestra.run.smithi157:> sudo hostname $(hostname -s) 2023-01-25T11:39:43.924 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-01-25T11:39:43.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo lsb_release -is 2023-01-25T11:39:43.972 INFO:teuthology.orchestra.run.smithi119.stdout:CentOSStream 2023-01-25T11:39:43.973 DEBUG:teuthology.misc:System to be installed: CentOSStream 2023-01-25T11:39:43.974 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:39:44.187 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '44e4095da12a8269eb6cfe163cbb0d00b1a6412c', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/67600/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-1986-g44e4095d', 'node_name': '172.21.2.6+braggi06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-1986.g44e4095d'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-01-24 17:44:14.942241', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-yuri6-testing-2023-01-24-0746', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-01-25T11:39:44.410 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-yuri6-testing-2023-01-24-0746, sha1 44e4095da12a8269eb6cfe163cbb0d00b1a6412c from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=44e4095da12a8269eb6cfe163cbb0d00b1a6412c 2023-01-25T11:39:44.411 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/x86_64/flavors/default/cephadm 2023-01-25T11:39:44.411 DEBUG:teuthology.orchestra.run.smithi119:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-01-25T11:39:45.068 INFO:teuthology.orchestra.run.smithi119.stdout:-rw-rw-r--. 1 ubuntu ubuntu 377960 Jan 25 11:39 /home/ubuntu/cephtest/cephadm 2023-01-25T11:39:45.069 DEBUG:teuthology.orchestra.run.smithi157:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-yuri6-testing-2023-01-24-0746/44e4095da12a8269eb6cfe163cbb0d00b1a6412c/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-01-25T11:39:45.574 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-rw-r--. 1 ubuntu ubuntu 377960 Jan 25 11:39 /home/ubuntu/cephtest/cephadm 2023-01-25T11:39:45.574 DEBUG:teuthology.orchestra.run.smithi119:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-01-25T11:39:45.597 DEBUG:teuthology.orchestra.run.smithi157:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-01-25T11:39:45.626 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c on all hosts... 2023-01-25T11:39:45.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c pull 2023-01-25T11:39:45.639 DEBUG:teuthology.orchestra.run.smithi157:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c pull 2023-01-25T11:39:47.005 INFO:teuthology.orchestra.run.smithi157.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c... 2023-01-25T11:39:47.148 INFO:teuthology.orchestra.run.smithi119.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c... 2023-01-25T11:40:11.274 INFO:teuthology.orchestra.run.smithi157.stdout:{ 2023-01-25T11:40:11.275 INFO:teuthology.orchestra.run.smithi157.stdout: "ceph_version": "ceph version 18.0.0-1986-g44e4095d (44e4095da12a8269eb6cfe163cbb0d00b1a6412c) reef (dev)", 2023-01-25T11:40:11.275 INFO:teuthology.orchestra.run.smithi157.stdout: "image_id": "b9ee3ec9b0ba1757c4b9e53705433eac5b03eac785836bd0991b1a42613d95a7", 2023-01-25T11:40:11.275 INFO:teuthology.orchestra.run.smithi157.stdout: "repo_digests": [ 2023-01-25T11:40:11.275 INFO:teuthology.orchestra.run.smithi157.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af0a1fa8aa875db45f97a762c2a395ca6fc313b318d65a9a19c9b62faa55a46f" 2023-01-25T11:40:11.275 INFO:teuthology.orchestra.run.smithi157.stdout: ] 2023-01-25T11:40:11.276 INFO:teuthology.orchestra.run.smithi157.stdout:} 2023-01-25T11:40:19.627 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-01-25T11:40:19.627 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.0.0-1986-g44e4095d (44e4095da12a8269eb6cfe163cbb0d00b1a6412c) reef (dev)", 2023-01-25T11:40:19.627 INFO:teuthology.orchestra.run.smithi119.stdout: "image_id": "b9ee3ec9b0ba1757c4b9e53705433eac5b03eac785836bd0991b1a42613d95a7", 2023-01-25T11:40:19.628 INFO:teuthology.orchestra.run.smithi119.stdout: "repo_digests": [ 2023-01-25T11:40:19.628 INFO:teuthology.orchestra.run.smithi119.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:af0a1fa8aa875db45f97a762c2a395ca6fc313b318d65a9a19c9b62faa55a46f" 2023-01-25T11:40:19.628 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-01-25T11:40:19.628 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-01-25T11:40:19.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph 2023-01-25T11:40:19.692 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph 2023-01-25T11:40:19.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /etc/ceph 2023-01-25T11:40:19.765 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /etc/ceph 2023-01-25T11:40:19.798 INFO:tasks.cephadm:Writing seed config... 2023-01-25T11:40:19.799 INFO:tasks.cephadm: override: [client] debug client = 20 2023-01-25T11:40:19.800 INFO:tasks.cephadm: override: [client] debug mgrc = 20 2023-01-25T11:40:19.800 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-01-25T11:40:19.800 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2023-01-25T11:40:19.800 INFO:tasks.cephadm: override: [mds] debug mgrc = 20 2023-01-25T11:40:19.801 INFO:tasks.cephadm: override: [mgr] debug client = 20 2023-01-25T11:40:19.801 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-01-25T11:40:19.801 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-01-25T11:40:19.801 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-01-25T11:40:19.802 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-01-25T11:40:19.802 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-01-25T11:40:19.802 INFO:tasks.cephadm: override: [osd] bluefs allocator = hybrid 2023-01-25T11:40:19.802 INFO:tasks.cephadm: override: [osd] bluestore allocator = hybrid 2023-01-25T11:40:19.803 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-01-25T11:40:19.803 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-01-25T11:40:19.803 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2023-01-25T11:40:19.803 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2023-01-25T11:40:19.804 INFO:tasks.cephadm: override: [osd] debug mgrc = 20 2023-01-25T11:40:19.804 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-01-25T11:40:19.804 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-01-25T11:40:19.804 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2023-01-25T11:40:19.805 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-01-25T11:40:19.805 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-01-25T11:40:19.805 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-01-25T11:40:19.805 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-01-25T11:40:19.806 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-01-25T11:40:19.806 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-01-25T11:40:19.807 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-01-25T11:40:19.826 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f64523de-9ca4-11ed-9e56-001a4aab830c mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true bluefs allocator = hybrid bluestore allocator = hybrid bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug mgrc = 20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug client = 20 debug mgrc = 20 debug ms = 1 [mds] debug mgrc = 20 2023-01-25T11:40:19.827 DEBUG:teuthology.orchestra.run.smithi119:mon.a> sudo journalctl -f -n 0 -u ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mon.a.service 2023-01-25T11:40:19.870 DEBUG:teuthology.orchestra.run.smithi119:mgr.a> sudo journalctl -f -n 0 -u ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mgr.a.service 2023-01-25T11:40:19.900 INFO:journalctl@ceph.mon.a.smithi119.stdout:-- Logs begin at Wed 2023-01-25 11:27:47 UTC. -- 2023-01-25T11:40:19.901 INFO:tasks.cephadm:Bootstrapping... 2023-01-25T11:40:19.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c -v bootstrap --fsid f64523de-9ca4-11ed-9e56-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id a --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.119 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-01-25T11:40:19.930 INFO:journalctl@ceph.mgr.a.smithi119.stdout:-- Logs begin at Wed 2023-01-25 11:27:47 UTC. -- 2023-01-25T11:40:20.151 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-01-25T11:40:20.151 INFO:teuthology.orchestra.run.smithi119.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c', '-v', 'bootstrap', '--fsid', 'f64523de-9ca4-11ed-9e56-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'a', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.119', '--skip-admin-label'] 2023-01-25T11:40:20.302 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.2.0 2023-01-25T11:40:20.302 INFO:teuthology.orchestra.run.smithi119.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-01-25T11:40:20.303 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying podman|docker is present... 2023-01-25T11:40:20.440 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.2.0 2023-01-25T11:40:20.441 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying lvm2 is present... 2023-01-25T11:40:20.441 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying time synchronization is in place... 2023-01-25T11:40:20.447 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-01-25T11:40:20.448 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-01-25T11:40:20.453 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-01-25T11:40:20.454 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-01-25T11:40:20.459 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2023-01-25T11:40:20.465 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2023-01-25T11:40:20.465 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2023-01-25T11:40:20.466 INFO:teuthology.orchestra.run.smithi119.stdout:Repeating the final host check... 2023-01-25T11:40:20.604 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout 4.2.0 2023-01-25T11:40:20.605 INFO:teuthology.orchestra.run.smithi119.stdout:podman (/bin/podman) version 4.2.0 is present 2023-01-25T11:40:20.605 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl is present 2023-01-25T11:40:20.605 INFO:teuthology.orchestra.run.smithi119.stdout:lvcreate is present 2023-01-25T11:40:20.610 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-01-25T11:40:20.611 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-01-25T11:40:20.617 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-01-25T11:40:20.617 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-01-25T11:40:20.622 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout enabled 2023-01-25T11:40:20.629 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout active 2023-01-25T11:40:20.629 INFO:teuthology.orchestra.run.smithi119.stdout:Unit chronyd.service is enabled and running 2023-01-25T11:40:20.629 INFO:teuthology.orchestra.run.smithi119.stdout:Host looks OK 2023-01-25T11:40:20.629 INFO:teuthology.orchestra.run.smithi119.stdout:Cluster fsid: f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:20.630 INFO:teuthology.orchestra.run.smithi119.stdout:Acquiring lock 140009743223272 on /run/cephadm/f64523de-9ca4-11ed-9e56-001a4aab830c.lock 2023-01-25T11:40:20.630 INFO:teuthology.orchestra.run.smithi119.stdout:Lock 140009743223272 acquired on /run/cephadm/f64523de-9ca4-11ed-9e56-001a4aab830c.lock 2023-01-25T11:40:20.630 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 3300 ... 2023-01-25T11:40:20.631 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying IP 172.21.15.119 port 6789 ... 2023-01-25T11:40:20.631 INFO:teuthology.orchestra.run.smithi119.stdout:Base mon IP(s) is [172.21.15.119:3300, 172.21.15.119:6789], mon addrv is [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2023-01-25T11:40:20.633 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.119 metric 100 2023-01-25T11:40:20.633 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.119 metric 100 2023-01-25T11:40:20.636 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-01-25T11:40:20.636 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2023-01-25T11:40:20.636 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1793sec hoplimit 64 pref medium 2023-01-25T11:40:20.638 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-01-25T11:40:20.639 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-01-25T11:40:20.639 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-01-25T11:40:20.639 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-01-25T11:40:20.639 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-01-25T11:40:20.640 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-01-25T11:40:20.640 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2023-01-25T11:40:20.641 INFO:teuthology.orchestra.run.smithi119.stdout:Mon IP `172.21.15.119` is in CIDR network `172.21.0.0/20` 2023-01-25T11:40:20.641 INFO:teuthology.orchestra.run.smithi119.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-01-25T11:40:20.641 INFO:teuthology.orchestra.run.smithi119.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-01-25T11:40:20.641 INFO:teuthology.orchestra.run.smithi119.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c... 2023-01-25T11:40:23.190 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stdout b9ee3ec9b0ba1757c4b9e53705433eac5b03eac785836bd0991b1a42613d95a7 2023-01-25T11:40:23.190 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44e4095da12a8269eb6cfe163cbb0d00b1a6412c... 2023-01-25T11:40:23.190 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Getting image source signatures 2023-01-25T11:40:23.191 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:a45ffd7b2c664ad26a4072a77702683e8fc7cff2b6f0fb540c4684c98da4e399 2023-01-25T11:40:23.191 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:aa16c48af92239fef3d4f8c28b3ad613ba91dcadb275f88623c48df79127d165 2023-01-25T11:40:23.191 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:53525f8b521a5f9317c9ce853653e176e9ed1037575ae7f5da2a2bed6a450060 2023-01-25T11:40:23.191 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:e2873b3fbbec402bafc59c62563e7264f20ad5dccce4dae4ebfb5edd55afb4cb 2023-01-25T11:40:23.191 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying blob sha256:317bd5f21d3fa946c4fb62d54870936260c660d455fde8674fba06ae80df1cf8 2023-01-25T11:40:23.192 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Copying config sha256:b9ee3ec9b0ba1757c4b9e53705433eac5b03eac785836bd0991b1a42613d95a7 2023-01-25T11:40:23.192 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Writing manifest to image destination 2023-01-25T11:40:23.192 INFO:teuthology.orchestra.run.smithi119.stdout:/bin/podman: stderr Storing signatures 2023-01-25T11:40:26.343 INFO:teuthology.orchestra.run.smithi119.stdout:ceph: stdout ceph version 18.0.0-1986-g44e4095d (44e4095da12a8269eb6cfe163cbb0d00b1a6412c) reef (dev) 2023-01-25T11:40:26.343 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph version: ceph version 18.0.0-1986-g44e4095d (44e4095da12a8269eb6cfe163cbb0d00b1a6412c) reef (dev) 2023-01-25T11:40:26.343 INFO:teuthology.orchestra.run.smithi119.stdout:Extracting ceph user uid/gid from container image... 2023-01-25T11:40:27.104 INFO:teuthology.orchestra.run.smithi119.stdout:stat: stdout 167 167 2023-01-25T11:40:27.104 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial keys... 2023-01-25T11:40:27.856 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQArFdFjX8VWGRAAejcW+co1XgM5QdKVrjYKqg== 2023-01-25T11:40:28.601 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQAsFdFj7gzuCBAARbslM5zeZ1zvYlXAl0Gnpw== 2023-01-25T11:40:29.243 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph-authtool: stdout AQAsFdFjEs+eMxAA7AAz4+pRznzrfPARCIHDZQ== 2023-01-25T11:40:29.244 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial monmap... 2023-01-25T11:40:29.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-01-25T11:40:29.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-01-25T11:40:29.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:29.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-01-25T11:40:29.988 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool for a [v2:172.21.15.119:3300,v1:172.21.15.119:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-01-25T11:40:29.988 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = pacific 2023-01-25T11:40:29.988 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: set fsid to f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:29.988 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-01-25T11:40:29.988 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:40:29.989 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mon... 2023-01-25T11:40:30.830 INFO:teuthology.orchestra.run.smithi119.stdout:create mon.a on 2023-01-25T11:40:30.934 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-01-25T11:40:31.027 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-01-25T11:40:31.122 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c.target → /etc/systemd/system/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c.target. 2023-01-25T11:40:31.122 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c.target → /etc/systemd/system/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c.target. 2023-01-25T11:40:31.244 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mon.a 2023-01-25T11:40:31.244 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mon.a.service: Unit ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mon.a.service not loaded. 2023-01-25T11:40:31.339 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c.target.wants/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@.service. 2023-01-25T11:40:32.249 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-01-25T11:40:32.249 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-01-25T11:40:32.257 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-01-25T11:40:32.257 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-01-25T11:40:32.257 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-01-25T11:40:32.257 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2023-01-25T11:40:32.258 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon to start... 2023-01-25T11:40:32.258 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mon... 2023-01-25T11:40:32.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:32 smithi119 ceph-mon[101794]: mon.a is new leader, mons a in quorum (ranks 0) 2023-01-25T11:40:32.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:32 smithi119 systemd[1]: Started Ceph mon.a for f64523de-9ca4-11ed-9e56-001a4aab830c. 2023-01-25T11:40:33.197 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout cluster: 2023-01-25T11:40:33.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout id: f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:33.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-01-25T11:40:33.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:33.204 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout services: 2023-01-25T11:40:33.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.484967s) 2023-01-25T11:40:33.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-01-25T11:40:33.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-01-25T11:40:33.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:33.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout data: 2023-01-25T11:40:33.205 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-01-25T11:40:33.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-01-25T11:40:33.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-01-25T11:40:33.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout pgs: 2023-01-25T11:40:33.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:33.206 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.636+0000 7f5506c5f700 1 Processor -- start 2023-01-25T11:40:33.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.636+0000 7f5506c5f700 1 -- start start 2023-01-25T11:40:33.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.636+0000 7f5506c5f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f5500106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:33.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.636+0000 7f5506c5f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5500077f50 con 0x7f55001048c0 2023-01-25T11:40:33.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f55049fb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f5500106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:33.207 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f55049fb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f5500106ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45836/0 (socket says 172.21.15.119:45836) 2023-01-25T11:40:33.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f55049fb700 1 -- 172.21.15.119:0/506991965 learned_addr learned my addr 172.21.15.119:0/506991965 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:33.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f55049fb700 1 -- 172.21.15.119:0/506991965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55001071e0 con 0x7f55001048c0 2023-01-25T11:40:33.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f55049fb700 1 --2- 172.21.15.119:0/506991965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f5500106ca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0098a0 tx=0x7f54fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999a803af649ab28 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:33.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f54f77fe700 1 -- 172.21.15.119:0/506991965 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54fc004160 con 0x7f55001048c0 2023-01-25T11:40:33.208 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f54f77fe700 1 -- 172.21.15.119:0/506991965 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc0042c0 con 0x7f55001048c0 2023-01-25T11:40:33.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.637+0000 7f54f77fe700 1 -- 172.21.15.119:0/506991965 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54fc0045a0 con 0x7f55001048c0 2023-01-25T11:40:33.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 -- 172.21.15.119:0/506991965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 msgr2=0x7f5500106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:33.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 --2- 172.21.15.119:0/506991965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f5500106ca0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0098a0 tx=0x7f54fc009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:33.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 -- 172.21.15.119:0/506991965 shutdown_connections 2023-01-25T11:40:33.209 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 --2- 172.21.15.119:0/506991965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f5500106ca0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:33.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 -- 172.21.15.119:0/506991965 >> 172.21.15.119:0/506991965 conn(0x7f55000fe220 msgr2=0x7f5500100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:33.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 -- 172.21.15.119:0/506991965 shutdown_connections 2023-01-25T11:40:33.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 -- 172.21.15.119:0/506991965 wait complete. 2023-01-25T11:40:33.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 Processor -- start 2023-01-25T11:40:33.210 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.638+0000 7f5506c5f700 1 -- start start 2023-01-25T11:40:33.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f5506c5f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f55001973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:33.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f5506c5f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55001978e0 con 0x7f55001048c0 2023-01-25T11:40:33.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f55049fb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f55001973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:33.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f55049fb700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f55001973a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45846/0 (socket says 172.21.15.119:45846) 2023-01-25T11:40:33.211 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f55049fb700 1 -- 172.21.15.119:0/3286693822 learned_addr learned my addr 172.21.15.119:0/3286693822 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:33.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f55049fb700 1 -- 172.21.15.119:0/3286693822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc009550 con 0x7f55001048c0 2023-01-25T11:40:33.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f55049fb700 1 --2- 172.21.15.119:0/3286693822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f55001973a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5500077280 tx=0x7f54fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:33.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f54f5ffb700 1 -- 172.21.15.119:0/3286693822 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54fc011630 con 0x7f55001048c0 2023-01-25T11:40:33.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f54f5ffb700 1 -- 172.21.15.119:0/3286693822 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54fc011790 con 0x7f55001048c0 2023-01-25T11:40:33.212 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.639+0000 7f54f5ffb700 1 -- 172.21.15.119:0/3286693822 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54fc011a30 con 0x7f55001048c0 2023-01-25T11:40:33.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5500107680 con 0x7f55001048c0 2023-01-25T11:40:33.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55001074f0 con 0x7f55001048c0 2023-01-25T11:40:33.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f54f5ffb700 1 -- 172.21.15.119:0/3286693822 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f54fc017020 con 0x7f55001048c0 2023-01-25T11:40:33.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f54f5ffb700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-01-25T11:40:33.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f54f5ffb700 4 mgrc handle_mgr_map Got map version 1 2023-01-25T11:40:33.213 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f54f5ffb700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:40:33.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f54f5ffb700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:40:33.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.640+0000 7f54f5ffb700 1 -- 172.21.15.119:0/3286693822 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f54fc01c450 con 0x7f55001048c0 2023-01-25T11:40:33.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.641+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5500107810 con 0x7f55001048c0 2023-01-25T11:40:33.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.643+0000 7f54f5ffb700 1 -- 172.21.15.119:0/3286693822 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f54fc01b450 con 0x7f55001048c0 2023-01-25T11:40:33.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.713+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5500191280 con 0x7f55001048c0 2023-01-25T11:40:33.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.713+0000 7f54f5ffb700 1 -- 172.21.15.119:0/3286693822 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f54fc01b5f0 con 0x7f55001048c0 2023-01-25T11:40:33.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 10 mgrc shutdown 2023-01-25T11:40:33.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 msgr2=0x7f55001973a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:33.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 1 --2- 172.21.15.119:0/3286693822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f55001973a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5500077280 tx=0x7f54fc004750 comp rx=0 tx=0).stop 2023-01-25T11:40:33.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 shutdown_connections 2023-01-25T11:40:33.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 1 --2- 172.21.15.119:0/3286693822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55001048c0 0x7f55001973a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:33.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 >> 172.21.15.119:0/3286693822 conn(0x7f55000fe220 msgr2=0x7f55000ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:33.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 shutdown_connections 2023-01-25T11:40:33.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:32.714+0000 7f5506c5f700 1 -- 172.21.15.119:0/3286693822 wait complete. 2023-01-25T11:40:33.216 INFO:teuthology.orchestra.run.smithi119.stdout:mon is available 2023-01-25T11:40:33.217 INFO:teuthology.orchestra.run.smithi119.stdout:Assimilating anything we can from ceph.conf... 2023-01-25T11:40:33.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:33 smithi119 ceph-mon[101794]: mon.a is new leader, mons a in quorum (ranks 0) 2023-01-25T11:40:33.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:33 smithi119 ceph-mon[101794]: monmap e1: 1 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} 2023-01-25T11:40:33.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:33 smithi119 ceph-mon[101794]: fsmap 2023-01-25T11:40:33.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:33 smithi119 ceph-mon[101794]: osdmap e1: 0 total, 0 up, 0 in 2023-01-25T11:40:33.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:33 smithi119 ceph-mon[101794]: mgrmap e1: no daemons active 2023-01-25T11:40:33.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:33 smithi119 ceph-mon[101794]: from='client.? 172.21.15.119:0/3286693822' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-01-25T11:40:34.143 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:34.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2023-01-25T11:40:34.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:34.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.119:3300,v1:172.21.15.119:6789] 2023-01-25T11:40:34.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-01-25T11:40:34.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-01-25T11:40:34.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-01-25T11:40:34.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-01-25T11:40:34.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:34.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2023-01-25T11:40:34.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-01-25T11:40:34.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:34.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2023-01-25T11:40:34.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-01-25T11:40:34.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-01-25T11:40:34.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-01-25T11:40:34.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-01-25T11:40:34.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-01-25T11:40:34.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.602+0000 7fca0a00e700 1 Processor -- start 2023-01-25T11:40:34.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.602+0000 7fca0a00e700 1 -- start start 2023-01-25T11:40:34.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.602+0000 7fca0a00e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca04102900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:34.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.602+0000 7fca0a00e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04102e40 con 0x7fca041024f0 2023-01-25T11:40:34.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.602+0000 7fca037fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca04102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:34.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.602+0000 7fca037fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca04102900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45848/0 (socket says 172.21.15.119:45848) 2023-01-25T11:40:34.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.602+0000 7fca037fe700 1 -- 172.21.15.119:0/3786470051 learned_addr learned my addr 172.21.15.119:0/3786470051 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:34.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca037fe700 1 -- 172.21.15.119:0/3786470051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca04102f80 con 0x7fca041024f0 2023-01-25T11:40:34.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca037fe700 1 --2- 172.21.15.119:0/3786470051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca04102900 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40098a0 tx=0x7fc9f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8f7df15aedbbad server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:34.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca027fc700 1 -- 172.21.15.119:0/3786470051 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f4004160 con 0x7fca041024f0 2023-01-25T11:40:34.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca027fc700 1 -- 172.21.15.119:0/3786470051 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f40042c0 con 0x7fca041024f0 2023-01-25T11:40:34.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca027fc700 1 -- 172.21.15.119:0/3786470051 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f40045a0 con 0x7fca041024f0 2023-01-25T11:40:34.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca0a00e700 1 -- 172.21.15.119:0/3786470051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 msgr2=0x7fca04102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:34.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca0a00e700 1 --2- 172.21.15.119:0/3786470051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca04102900 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40098a0 tx=0x7fc9f4009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:34.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca0a00e700 1 -- 172.21.15.119:0/3786470051 shutdown_connections 2023-01-25T11:40:34.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca0a00e700 1 --2- 172.21.15.119:0/3786470051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca04102900 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:34.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.603+0000 7fca0a00e700 1 -- 172.21.15.119:0/3786470051 >> 172.21.15.119:0/3786470051 conn(0x7fca040fdaf0 msgr2=0x7fca040fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:34.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.604+0000 7fca0a00e700 1 -- 172.21.15.119:0/3786470051 shutdown_connections 2023-01-25T11:40:34.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.604+0000 7fca0a00e700 1 -- 172.21.15.119:0/3786470051 wait complete. 2023-01-25T11:40:34.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.604+0000 7fca0a00e700 1 Processor -- start 2023-01-25T11:40:34.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.604+0000 7fca0a00e700 1 -- start start 2023-01-25T11:40:34.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.604+0000 7fca0a00e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca041972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:34.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.604+0000 7fca0a00e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca04197810 con 0x7fca041024f0 2023-01-25T11:40:34.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca037fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca041972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:34.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca037fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca041972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45862/0 (socket says 172.21.15.119:45862) 2023-01-25T11:40:34.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca037fe700 1 -- 172.21.15.119:0/2206815175 learned_addr learned my addr 172.21.15.119:0/2206815175 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:34.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca037fe700 1 -- 172.21.15.119:0/2206815175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f4009550 con 0x7fca041024f0 2023-01-25T11:40:34.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca037fe700 1 --2- 172.21.15.119:0/2206815175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca041972d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009fb0 tx=0x7fc9f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:34.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f4003dd0 con 0x7fca041024f0 2023-01-25T11:40:34.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f4011630 con 0x7fca041024f0 2023-01-25T11:40:34.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.605+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc9f4011910 con 0x7fca041024f0 2023-01-25T11:40:34.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca04103a30 con 0x7fca041024f0 2023-01-25T11:40:34.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca041038a0 con 0x7fca041024f0 2023-01-25T11:40:34.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc9f402b020 con 0x7fca041024f0 2023-01-25T11:40:34.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca00ff9700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-01-25T11:40:34.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca00ff9700 4 mgrc handle_mgr_map Got map version 1 2023-01-25T11:40:34.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca00ff9700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:40:34.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca00ff9700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:40:34.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc9f401a790 con 0x7fca041024f0 2023-01-25T11:40:34.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.606+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca04103bc0 con 0x7fca041024f0 2023-01-25T11:40:34.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.609+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7fc9f401cd20 con 0x7fca041024f0 2023-01-25T11:40:34.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.676+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fca04191190 con 0x7fca041024f0 2023-01-25T11:40:34.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.679+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 7 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fc9f401ae40 con 0x7fca041024f0 2023-01-25T11:40:34.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.679+0000 7fca00ff9700 1 -- 172.21.15.119:0/2206815175 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fc9f4025790 con 0x7fca041024f0 2023-01-25T11:40:34.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 10 mgrc shutdown 2023-01-25T11:40:34.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 msgr2=0x7fca041972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:34.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 1 --2- 172.21.15.119:0/2206815175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca041972d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009fb0 tx=0x7fc9f4004750 comp rx=0 tx=0).stop 2023-01-25T11:40:34.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 shutdown_connections 2023-01-25T11:40:34.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 1 --2- 172.21.15.119:0/2206815175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca041024f0 0x7fca041972d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:34.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 >> 172.21.15.119:0/2206815175 conn(0x7fca040fdaf0 msgr2=0x7fca04189f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:34.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 shutdown_connections 2023-01-25T11:40:34.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:33.680+0000 7fca0a00e700 1 -- 172.21.15.119:0/2206815175 wait complete. 2023-01-25T11:40:34.157 INFO:teuthology.orchestra.run.smithi119.stdout:Generating new minimal ceph.conf... 2023-01-25T11:40:34.385 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:34 smithi119 ceph-mon[101794]: from='client.? 172.21.15.119:0/2206815175' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-01-25T11:40:34.386 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:34 smithi119 ceph-mon[101794]: from='client.? 172.21.15.119:0/2206815175' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-01-25T11:40:35.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.530+0000 7ff400ae0700 1 Processor -- start 2023-01-25T11:40:35.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.530+0000 7ff400ae0700 1 -- start start 2023-01-25T11:40:35.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.530+0000 7ff400ae0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:35.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.530+0000 7ff400ae0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc077f70 con 0x7ff3fc1048a0 2023-01-25T11:40:35.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.530+0000 7ff3fa59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.531+0000 7ff3fa59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45868/0 (socket says 172.21.15.119:45868) 2023-01-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.531+0000 7ff3fa59c700 1 -- 172.21.15.119:0/3651731943 learned_addr learned my addr 172.21.15.119:0/3651731943 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.531+0000 7ff3fa59c700 1 -- 172.21.15.119:0/3651731943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc1071c0 con 0x7ff3fc1048a0 2023-01-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.531+0000 7ff3fa59c700 1 --2- 172.21.15.119:0/3651731943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc106c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0098a0 tx=0x7ff3ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b36a2a2030415279 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:35.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.531+0000 7ff3f9d9b700 1 -- 172.21.15.119:0/3651731943 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3ec004160 con 0x7ff3fc1048a0 2023-01-25T11:40:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.531+0000 7ff3f9d9b700 1 -- 172.21.15.119:0/3651731943 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7ff3ec0036a0 con 0x7ff3fc1048a0 2023-01-25T11:40:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.531+0000 7ff3f9d9b700 1 -- 172.21.15.119:0/3651731943 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3ec011870 con 0x7ff3fc1048a0 2023-01-25T11:40:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 -- 172.21.15.119:0/3651731943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 msgr2=0x7ff3fc106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 --2- 172.21.15.119:0/3651731943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc106c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0098a0 tx=0x7ff3ec009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:35.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 -- 172.21.15.119:0/3651731943 shutdown_connections 2023-01-25T11:40:35.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 --2- 172.21.15.119:0/3651731943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc106c80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:35.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 -- 172.21.15.119:0/3651731943 >> 172.21.15.119:0/3651731943 conn(0x7ff3fc0fe220 msgr2=0x7ff3fc100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:35.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 -- 172.21.15.119:0/3651731943 shutdown_connections 2023-01-25T11:40:35.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 -- 172.21.15.119:0/3651731943 wait complete. 2023-01-25T11:40:35.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.532+0000 7ff400ae0700 1 Processor -- start 2023-01-25T11:40:35.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff400ae0700 1 -- start start 2023-01-25T11:40:35.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff400ae0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:35.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff400ae0700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc198080 con 0x7ff3fc1048a0 2023-01-25T11:40:35.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3fa59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:35.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3fa59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45878/0 (socket says 172.21.15.119:45878) 2023-01-25T11:40:35.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3fa59c700 1 -- 172.21.15.119:0/2825158918 learned_addr learned my addr 172.21.15.119:0/2825158918 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:35.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3fa59c700 1 -- 172.21.15.119:0/2825158918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec009550 con 0x7ff3fc1048a0 2023-01-25T11:40:35.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3fa59c700 1 --2- 172.21.15.119:0/2825158918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc197b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0119c0 tx=0x7ff3ec011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:35.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3f3fff700 1 -- 172.21.15.119:0/2825158918 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3ec004110 con 0x7ff3fc1048a0 2023-01-25T11:40:35.189 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3f3fff700 1 -- 172.21.15.119:0/2825158918 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7ff3ec01b440 con 0x7ff3fc1048a0 2023-01-25T11:40:35.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.533+0000 7ff3f3fff700 1 -- 172.21.15.119:0/2825158918 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff3ec023400 con 0x7ff3fc1048a0 2023-01-25T11:40:35.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc108150 con 0x7ff3fc1048a0 2023-01-25T11:40:35.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc107fc0 con 0x7ff3fc1048a0 2023-01-25T11:40:35.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff3f3fff700 1 -- 172.21.15.119:0/2825158918 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff3ec017020 con 0x7ff3fc1048a0 2023-01-25T11:40:35.190 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff3f3fff700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-01-25T11:40:35.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff3f3fff700 4 mgrc handle_mgr_map Got map version 1 2023-01-25T11:40:35.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff3f3fff700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:40:35.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff3f3fff700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:40:35.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.534+0000 7ff3f3fff700 1 -- 172.21.15.119:0/2825158918 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff3ec023c60 con 0x7ff3fc1048a0 2023-01-25T11:40:35.191 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.535+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc1082e0 con 0x7ff3fc1048a0 2023-01-25T11:40:35.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.537+0000 7ff3f3fff700 1 -- 172.21.15.119:0/2825158918 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7ff3ec004270 con 0x7ff3fc1048a0 2023-01-25T11:40:35.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.604+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff3fc061b00 con 0x7ff3fc1048a0 2023-01-25T11:40:35.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.604+0000 7ff3f3fff700 1 -- 172.21.15.119:0/2825158918 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7ff3ec004410 con 0x7ff3fc1048a0 2023-01-25T11:40:35.192 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 10 mgrc shutdown 2023-01-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 msgr2=0x7ff3fc197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 1 --2- 172.21.15.119:0/2825158918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc197b40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec0119c0 tx=0x7ff3ec011b30 comp rx=0 tx=0).stop 2023-01-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 shutdown_connections 2023-01-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 1 --2- 172.21.15.119:0/2825158918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc1048a0 0x7ff3fc197b40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 >> 172.21.15.119:0/2825158918 conn(0x7ff3fc0fe220 msgr2=0x7ff3fc0ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:35.193 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 shutdown_connections 2023-01-25T11:40:35.194 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:34.605+0000 7ff400ae0700 1 -- 172.21.15.119:0/2825158918 wait complete. 2023-01-25T11:40:35.194 INFO:teuthology.orchestra.run.smithi119.stdout:Restarting the monitor... 2023-01-25T11:40:35.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:35 smithi119 systemd[1]: Stopping Ceph mon.a for f64523de-9ca4-11ed-9e56-001a4aab830c... 2023-01-25T11:40:35.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:35 smithi119 ceph-mon[101794]: from='client.? 172.21.15.119:0/2825158918' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-01-25T11:40:35.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:35 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mon-a[101790]: 2023-01-25T11:40:35.494+0000 7f9b99b24700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-01-25T11:40:35.861 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:35 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mon-a[101790]: 2023-01-25T11:40:35.494+0000 7f9b99b24700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-01-25T11:40:36.111 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:35 smithi119 bash[102053]: ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mon-a 2023-01-25T11:40:36.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:36 smithi119 systemd[1]: ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mon.a.service: Succeeded. 2023-01-25T11:40:36.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:36 smithi119 systemd[1]: Stopped Ceph mon.a for f64523de-9ca4-11ed-9e56-001a4aab830c. 2023-01-25T11:40:36.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:36 smithi119 systemd[1]: Starting Ceph mon.a for f64523de-9ca4-11ed-9e56-001a4aab830c... 2023-01-25T11:40:37.032 INFO:teuthology.orchestra.run.smithi119.stdout:Setting mon public_network to 172.21.0.0/20 2023-01-25T11:40:37.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:36 smithi119 podman[102143]: 2023-01-25T11:40:37.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:36 smithi119 bash[102143]: e0abe03e453e9efc18a54148efe47432045e7d3a90e15056459c3fa1ae2702f1 2023-01-25T11:40:37.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: set uid:gid to 167:167 (ceph:ceph) 2023-01-25T11:40:37.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: ceph version 18.0.0-1986-g44e4095d (44e4095da12a8269eb6cfe163cbb0d00b1a6412c) reef (dev), process ceph-mon, pid 2 2023-01-25T11:40:37.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: pidfile_write: ignore empty --pid-file 2023-01-25T11:40:37.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: load: jerasure load: lrc 2023-01-25T11:40:37.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: RocksDB version: 6.15.5 2023-01-25T11:40:37.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2023-01-25T11:40:37.142 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Compile date Jan 24 2023 2023-01-25T11:40:37.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: DB SUMMARY 2023-01-25T11:40:37.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: DB Session ID: EV6JGHAIAQH245QV4WNL 2023-01-25T11:40:37.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: CURRENT file: CURRENT 2023-01-25T11:40:37.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: IDENTITY file: IDENTITY 2023-01-25T11:40:37.143 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2023-01-25T11:40:37.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-01-25T11:40:37.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 89718 ; 2023-01-25T11:40:37.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.error_if_exists: 0 2023-01-25T11:40:37.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.create_if_missing: 0 2023-01-25T11:40:37.144 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.paranoid_checks: 1 2023-01-25T11:40:37.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-01-25T11:40:37.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.env: 0x5586556013a0 2023-01-25T11:40:37.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.fs: Posix File System 2023-01-25T11:40:37.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.info_log: 0x558655fd0360 2023-01-25T11:40:37.145 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_file_opening_threads: 16 2023-01-25T11:40:37.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.statistics: (nil) 2023-01-25T11:40:37.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.use_fsync: 0 2023-01-25T11:40:37.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_log_file_size: 0 2023-01-25T11:40:37.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-01-25T11:40:37.146 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.log_file_time_to_roll: 0 2023-01-25T11:40:37.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.keep_log_file_num: 1000 2023-01-25T11:40:37.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.recycle_log_file_num: 0 2023-01-25T11:40:37.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.allow_fallocate: 1 2023-01-25T11:40:37.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.allow_mmap_reads: 0 2023-01-25T11:40:37.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.allow_mmap_writes: 0 2023-01-25T11:40:37.147 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.use_direct_reads: 0 2023-01-25T11:40:37.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-01-25T11:40:37.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.create_missing_column_families: 0 2023-01-25T11:40:37.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.db_log_dir: 2023-01-25T11:40:37.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2023-01-25T11:40:37.148 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.table_cache_numshardbits: 6 2023-01-25T11:40:37.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.WAL_ttl_seconds: 0 2023-01-25T11:40:37.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.WAL_size_limit_MB: 0 2023-01-25T11:40:37.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-01-25T11:40:37.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-01-25T11:40:37.149 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.is_fd_close_on_exec: 1 2023-01-25T11:40:37.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.advise_random_on_open: 1 2023-01-25T11:40:37.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.db_write_buffer_size: 0 2023-01-25T11:40:37.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.write_buffer_manager: 0x5586561eaea0 2023-01-25T11:40:37.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-01-25T11:40:37.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2023-01-25T11:40:37.150 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-01-25T11:40:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.use_adaptive_mutex: 0 2023-01-25T11:40:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.rate_limiter: (nil) 2023-01-25T11:40:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-01-25T11:40:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.wal_recovery_mode: 2 2023-01-25T11:40:37.151 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.enable_thread_tracking: 0 2023-01-25T11:40:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.enable_pipelined_write: 0 2023-01-25T11:40:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.unordered_write: 0 2023-01-25T11:40:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-01-25T11:40:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-01-25T11:40:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-01-25T11:40:37.152 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-01-25T11:40:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.row_cache: None 2023-01-25T11:40:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.wal_filter: None 2023-01-25T11:40:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-01-25T11:40:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.allow_ingest_behind: 0 2023-01-25T11:40:37.153 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.preserve_deletes: 0 2023-01-25T11:40:37.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.two_write_queues: 0 2023-01-25T11:40:37.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.manual_wal_flush: 0 2023-01-25T11:40:37.154 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.atomic_flush: 0 2023-01-25T11:40:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-01-25T11:40:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.persist_stats_to_disk: 0 2023-01-25T11:40:37.155 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.write_dbid_to_manifest: 0 2023-01-25T11:40:37.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.log_readahead_size: 0 2023-01-25T11:40:37.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-01-25T11:40:37.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.best_efforts_recovery: 0 2023-01-25T11:40:37.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-01-25T11:40:37.156 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-01-25T11:40:37.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.allow_data_in_errors: 0 2023-01-25T11:40:37.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.db_host_id: __hostname__ 2023-01-25T11:40:37.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_background_jobs: 2 2023-01-25T11:40:37.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_background_compactions: -1 2023-01-25T11:40:37.157 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_subcompactions: 1 2023-01-25T11:40:37.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-01-25T11:40:37.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-01-25T11:40:37.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.delayed_write_rate : 16777216 2023-01-25T11:40:37.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_total_wal_size: 0 2023-01-25T11:40:37.158 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-01-25T11:40:37.159 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.stats_dump_period_sec: 600 2023-01-25T11:40:37.159 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.stats_persist_period_sec: 600 2023-01-25T11:40:37.159 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-01-25T11:40:37.159 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_open_files: -1 2023-01-25T11:40:37.160 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bytes_per_sync: 0 2023-01-25T11:40:37.160 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.wal_bytes_per_sync: 0 2023-01-25T11:40:37.160 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.strict_bytes_per_sync: 0 2023-01-25T11:40:37.160 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_readahead_size: 0 2023-01-25T11:40:37.160 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_background_flushes: -1 2023-01-25T11:40:37.161 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Compression algorithms supported: 2023-01-25T11:40:37.161 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-01-25T11:40:37.161 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kZSTD supported: 0 2023-01-25T11:40:37.161 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kXpressCompression supported: 0 2023-01-25T11:40:37.161 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kLZ4HCCompression supported: 1 2023-01-25T11:40:37.162 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kZlibCompression supported: 1 2023-01-25T11:40:37.163 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kSnappyCompression supported: 1 2023-01-25T11:40:37.163 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kLZ4Compression supported: 1 2023-01-25T11:40:37.164 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: kBZip2Compression supported: 0 2023-01-25T11:40:37.164 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Fast CRC32 supported: Supported on x86 2023-01-25T11:40:37.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2023-01-25T11:40:37.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2023-01-25T11:40:37.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-01-25T11:40:37.165 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.merge_operator: 2023-01-25T11:40:37.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_filter: None 2023-01-25T11:40:37.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_filter_factory: None 2023-01-25T11:40:37.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.sst_partitioner_factory: None 2023-01-25T11:40:37.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.memtable_factory: SkipListFactory 2023-01-25T11:40:37.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.table_factory: BlockBasedTable 2023-01-25T11:40:37.166 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558655ec9b68) 2023-01-25T11:40:37.167 INFO:journalctl@ceph.mon.a.smithi119.stdout: cache_index_and_filter_blocks: 1 2023-01-25T11:40:37.167 INFO:journalctl@ceph.mon.a.smithi119.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-01-25T11:40:37.168 INFO:journalctl@ceph.mon.a.smithi119.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-01-25T11:40:37.168 INFO:journalctl@ceph.mon.a.smithi119.stdout: pin_top_level_index_and_filter: 1 2023-01-25T11:40:37.169 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_type: 0 2023-01-25T11:40:37.169 INFO:journalctl@ceph.mon.a.smithi119.stdout: data_block_index_type: 0 2023-01-25T11:40:37.169 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_shortening: 1 2023-01-25T11:40:37.169 INFO:journalctl@ceph.mon.a.smithi119.stdout: data_block_hash_table_util_ratio: 0.750000 2023-01-25T11:40:37.169 INFO:journalctl@ceph.mon.a.smithi119.stdout: hash_index_allow_collision: 1 2023-01-25T11:40:37.170 INFO:journalctl@ceph.mon.a.smithi119.stdout: checksum: 1 2023-01-25T11:40:37.170 INFO:journalctl@ceph.mon.a.smithi119.stdout: no_block_cache: 0 2023-01-25T11:40:37.170 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache: 0x558655f35350 2023-01-25T11:40:37.170 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_name: BinnedLRUCache 2023-01-25T11:40:37.170 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_options: 2023-01-25T11:40:37.171 INFO:journalctl@ceph.mon.a.smithi119.stdout: capacity : 536870912 2023-01-25T11:40:37.171 INFO:journalctl@ceph.mon.a.smithi119.stdout: num_shard_bits : 4 2023-01-25T11:40:37.171 INFO:journalctl@ceph.mon.a.smithi119.stdout: strict_capacity_limit : 0 2023-01-25T11:40:37.171 INFO:journalctl@ceph.mon.a.smithi119.stdout: high_pri_pool_ratio: 0.000 2023-01-25T11:40:37.171 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_cache_compressed: (nil) 2023-01-25T11:40:37.172 INFO:journalctl@ceph.mon.a.smithi119.stdout: persistent_cache: (nil) 2023-01-25T11:40:37.172 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_size: 4096 2023-01-25T11:40:37.172 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_size_deviation: 10 2023-01-25T11:40:37.172 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_restart_interval: 16 2023-01-25T11:40:37.172 INFO:journalctl@ceph.mon.a.smithi119.stdout: index_block_restart_interval: 1 2023-01-25T11:40:37.173 INFO:journalctl@ceph.mon.a.smithi119.stdout: metadata_block_size: 4096 2023-01-25T11:40:37.173 INFO:journalctl@ceph.mon.a.smithi119.stdout: partition_filters: 0 2023-01-25T11:40:37.173 INFO:journalctl@ceph.mon.a.smithi119.stdout: use_delta_encoding: 1 2023-01-25T11:40:37.173 INFO:journalctl@ceph.mon.a.smithi119.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2023-01-25T11:40:37.173 INFO:journalctl@ceph.mon.a.smithi119.stdout: whole_key_filtering: 1 2023-01-25T11:40:37.174 INFO:journalctl@ceph.mon.a.smithi119.stdout: verify_compression: 0 2023-01-25T11:40:37.174 INFO:journalctl@ceph.mon.a.smithi119.stdout: read_amp_bytes_per_bit: 0 2023-01-25T11:40:37.174 INFO:journalctl@ceph.mon.a.smithi119.stdout: format_version: 4 2023-01-25T11:40:37.174 INFO:journalctl@ceph.mon.a.smithi119.stdout: enable_index_compression: 1 2023-01-25T11:40:37.174 INFO:journalctl@ceph.mon.a.smithi119.stdout: block_align: 0 2023-01-25T11:40:37.175 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.write_buffer_size: 33554432 2023-01-25T11:40:37.175 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_write_buffer_number: 2 2023-01-25T11:40:37.175 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression: NoCompression 2023-01-25T11:40:37.175 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression: Disabled 2023-01-25T11:40:37.175 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.prefix_extractor: nullptr 2023-01-25T11:40:37.176 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-01-25T11:40:37.176 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.num_levels: 7 2023-01-25T11:40:37.176 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-01-25T11:40:37.176 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-01-25T11:40:37.176 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-01-25T11:40:37.177 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-01-25T11:40:37.177 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-01-25T11:40:37.177 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-01-25T11:40:37.177 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-01-25T11:40:37.177 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-01-25T11:40:37.177 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-01-25T11:40:37.178 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-01-25T11:40:37.178 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression_opts.window_bits: -14 2023-01-25T11:40:37.178 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression_opts.level: 32767 2023-01-25T11:40:37.178 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression_opts.strategy: 0 2023-01-25T11:40:37.178 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-01-25T11:40:37.179 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-01-25T11:40:37.179 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-01-25T11:40:37.179 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compression_opts.enabled: false 2023-01-25T11:40:37.179 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-01-25T11:40:37.179 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-01-25T11:40:37.180 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-01-25T11:40:37.180 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.target_file_size_base: 67108864 2023-01-25T11:40:37.180 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.target_file_size_multiplier: 1 2023-01-25T11:40:37.180 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-01-25T11:40:37.180 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-01-25T11:40:37.181 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-01-25T11:40:37.181 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-01-25T11:40:37.181 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-01-25T11:40:37.181 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-01-25T11:40:37.181 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-01-25T11:40:37.182 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-01-25T11:40:37.182 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-01-25T11:40:37.182 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-01-25T11:40:37.182 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-01-25T11:40:37.182 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-01-25T11:40:37.182 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.arena_block_size: 4194304 2023-01-25T11:40:37.183 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-01-25T11:40:37.183 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-01-25T11:40:37.183 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2023-01-25T11:40:37.183 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.disable_auto_compactions: 0 2023-01-25T11:40:37.183 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-01-25T11:40:37.183 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-01-25T11:40:37.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-01-25T11:40:37.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-01-25T11:40:37.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-01-25T11:40:37.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-01-25T11:40:37.184 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-01-25T11:40:37.185 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-01-25T11:40:37.185 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-01-25T11:40:37.185 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-01-25T11:40:37.185 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.table_properties_collectors: 2023-01-25T11:40:37.185 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.inplace_update_support: 0 2023-01-25T11:40:37.186 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.inplace_update_num_locks: 10000 2023-01-25T11:40:37.186 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-01-25T11:40:37.186 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-01-25T11:40:37.186 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.memtable_huge_page_size: 0 2023-01-25T11:40:37.186 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.bloom_locality: 0 2023-01-25T11:40:37.187 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.max_successive_merges: 0 2023-01-25T11:40:37.187 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.optimize_filters_for_hits: 0 2023-01-25T11:40:37.187 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.paranoid_file_checks: 0 2023-01-25T11:40:37.187 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.force_consistency_checks: 1 2023-01-25T11:40:37.187 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.report_bg_io_stats: 0 2023-01-25T11:40:37.188 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.ttl: 2592000 2023-01-25T11:40:37.188 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.periodic_compaction_seconds: 0 2023-01-25T11:40:37.188 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.enable_blob_files: false 2023-01-25T11:40:37.188 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.min_blob_size: 0 2023-01-25T11:40:37.188 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.blob_file_size: 268435456 2023-01-25T11:40:37.189 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.blob_compression_type: NoCompression 2023-01-25T11:40:37.189 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.enable_blob_garbage_collection: false 2023-01-25T11:40:37.189 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-01-25T11:40:37.189 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-01-25T11:40:37.189 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2023-01-25T11:40:37.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2023-01-25T11:40:37.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: EVENT_LOG_v1 {"time_micros": 1674646837012505, "job": 1, "event": "recovery_started", "wal_files": [10]} 2023-01-25T11:40:37.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2023-01-25T11:40:37.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2023-01-25T11:40:37.190 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: EVENT_LOG_v1 {"time_micros": 1674646837013487, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 85738, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 83862, "index_size": 236, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 11420, "raw_average_key_size": 48, "raw_value_size": 77576, "raw_average_value_size": 327, "num_data_blocks": 10, "num_entries": 237, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1674646837, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "cba21677-afd4-42e8-b94c-20b2ddbf2c23", "db_session_id": "EV6JGHAIAQH245QV4WNL"}} 2023-01-25T11:40:37.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2023-01-25T11:40:37.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: EVENT_LOG_v1 {"time_micros": 1674646837013960, "job": 1, "event": "recovery_finished"} 2023-01-25T11:40:37.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-01-25T11:40:37.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x558655f1b880 2023-01-25T11:40:37.191 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: DB pointer 0x558655f4e000 2023-01-25T11:40:37.192 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2023-01-25T11:40:37.192 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: rocksdb: [db/db_impl/db_impl.cc:903] 2023-01-25T11:40:37.192 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** DB Stats ** 2023-01-25T11:40:37.192 INFO:journalctl@ceph.mon.a.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-01-25T11:40:37.192 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-01-25T11:40:37.193 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-01-25T11:40:37.193 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-01-25T11:40:37.193 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-01-25T11:40:37.193 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2023-01-25T11:40:37.193 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-01-25T11:40:37.193 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-01-25T11:40:37.194 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2023-01-25T11:40:37.194 INFO:journalctl@ceph.mon.a.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-01-25T11:40:37.194 INFO:journalctl@ceph.mon.a.smithi119.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-01-25T11:40:37.194 INFO:journalctl@ceph.mon.a.smithi119.stdout: L0 2/0 85.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 2023-01-25T11:40:37.194 INFO:journalctl@ceph.mon.a.smithi119.stdout: Sum 2/0 85.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 2023-01-25T11:40:37.195 INFO:journalctl@ceph.mon.a.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 2023-01-25T11:40:37.195 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-01-25T11:40:37.195 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2023-01-25T11:40:37.195 INFO:journalctl@ceph.mon.a.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-01-25T11:40:37.195 INFO:journalctl@ceph.mon.a.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-01-25T11:40:37.196 INFO:journalctl@ceph.mon.a.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.6 0.00 0.00 1 0.001 0 0 2023-01-25T11:40:37.196 INFO:journalctl@ceph.mon.a.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-01-25T11:40:37.196 INFO:journalctl@ceph.mon.a.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-01-25T11:40:37.196 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-01-25T11:40:37.196 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-01-25T11:40:37.197 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-01-25T11:40:37.197 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2023-01-25T11:40:37.197 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative compaction: 0.00 GB write, 19.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-01-25T11:40:37.197 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval compaction: 0.00 GB write, 19.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-01-25T11:40:37.197 INFO:journalctl@ceph.mon.a.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-01-25T11:40:37.198 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-01-25T11:40:37.198 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2023-01-25T11:40:37.198 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-01-25T11:40:37.198 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2023-01-25T11:40:37.198 INFO:journalctl@ceph.mon.a.smithi119.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-01-25T11:40:37.199 INFO:journalctl@ceph.mon.a.smithi119.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-01-25T11:40:37.199 INFO:journalctl@ceph.mon.a.smithi119.stdout: L0 2/0 85.49 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 2023-01-25T11:40:37.199 INFO:journalctl@ceph.mon.a.smithi119.stdout: Sum 2/0 85.49 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 127.6 0.00 0.00 1 0.001 0 0 2023-01-25T11:40:37.199 INFO:journalctl@ceph.mon.a.smithi119.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2023-01-25T11:40:37.199 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-01-25T11:40:37.199 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** Compaction Stats [default] ** 2023-01-25T11:40:37.200 INFO:journalctl@ceph.mon.a.smithi119.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2023-01-25T11:40:37.200 INFO:journalctl@ceph.mon.a.smithi119.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-01-25T11:40:37.200 INFO:journalctl@ceph.mon.a.smithi119.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 127.6 0.00 0.00 1 0.001 0 0 2023-01-25T11:40:37.200 INFO:journalctl@ceph.mon.a.smithi119.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-01-25T11:40:37.200 INFO:journalctl@ceph.mon.a.smithi119.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-01-25T11:40:37.201 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-01-25T11:40:37.201 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-01-25T11:40:37.201 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-01-25T11:40:37.201 INFO:journalctl@ceph.mon.a.smithi119.stdout: AddFile(Keys): cumulative 0, interval 0 2023-01-25T11:40:37.201 INFO:journalctl@ceph.mon.a.smithi119.stdout: Cumulative compaction: 0.00 GB write, 19.76 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-01-25T11:40:37.202 INFO:journalctl@ceph.mon.a.smithi119.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-01-25T11:40:37.202 INFO:journalctl@ceph.mon.a.smithi119.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-01-25T11:40:37.202 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-01-25T11:40:37.202 INFO:journalctl@ceph.mon.a.smithi119.stdout: ** File Read Latency Histogram By Level [default] ** 2023-01-25T11:40:37.202 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: starting mon.a rank 0 at public addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] at bind addrs [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:37.203 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???) e1 preinit fsid f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:37.203 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???).mds e1 new map 2023-01-25T11:40:37.203 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???).mds e1 print_map 2023-01-25T11:40:37.203 INFO:journalctl@ceph.mon.a.smithi119.stdout: e1 2023-01-25T11:40:37.203 INFO:journalctl@ceph.mon.a.smithi119.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-01-25T11:40:37.204 INFO:journalctl@ceph.mon.a.smithi119.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-01-25T11:40:37.204 INFO:journalctl@ceph.mon.a.smithi119.stdout: legacy client fscid: -1 2023-01-25T11:40:37.204 INFO:journalctl@ceph.mon.a.smithi119.stdout: 2023-01-25T11:40:37.204 INFO:journalctl@ceph.mon.a.smithi119.stdout: No filesystems configured 2023-01-25T11:40:37.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-01-25T11:40:37.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-01-25T11:40:37.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-01-25T11:40:37.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-01-25T11:40:37.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-01-25T11:40:37.205 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expand map: {default=false} 2023-01-25T11:40:37.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta from 'false' to 'false' 2023-01-25T11:40:37.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expanded map: {default=false} 2023-01-25T11:40:37.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expand map: {default=info} 2023-01-25T11:40:37.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta from 'info' to 'info' 2023-01-25T11:40:37.206 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expanded map: {default=info} 2023-01-25T11:40:37.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expand map: {default=daemon} 2023-01-25T11:40:37.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta from 'daemon' to 'daemon' 2023-01-25T11:40:37.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expanded map: {default=daemon} 2023-01-25T11:40:37.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expand map: {default=debug} 2023-01-25T11:40:37.207 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta from 'debug' to 'debug' 2023-01-25T11:40:37.208 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: expand_channel_meta expanded map: {default=debug} 2023-01-25T11:40:37.208 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 systemd[1]: Started Ceph mon.a for f64523de-9ca4-11ed-9e56-001a4aab830c. 2023-01-25T11:40:37.208 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mon.a is new leader, mons a in quorum (ranks 0) 2023-01-25T11:40:37.208 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: monmap e1: 1 mons at {a=[v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0]} removed_ranks: {} 2023-01-25T11:40:37.208 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: fsmap 2023-01-25T11:40:37.208 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: osdmap e1: 0 total, 0 up, 0 in 2023-01-25T11:40:37.209 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:37 smithi119 ceph-mon[102162]: mgrmap e1: no daemons active 2023-01-25T11:40:37.976 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.481+0000 7f0e36261700 1 Processor -- start 2023-01-25T11:40:37.977 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.481+0000 7f0e36261700 1 -- start start 2023-01-25T11:40:37.977 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.481+0000 7f0e36261700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e30106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:37.977 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.481+0000 7f0e36261700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e30107160 con 0x7f0e30104840 2023-01-25T11:40:37.977 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.482+0000 7f0e2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e30106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:37.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.482+0000 7f0e2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e30106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39146/0 (socket says 172.21.15.119:39146) 2023-01-25T11:40:37.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.482+0000 7f0e2f7fe700 1 -- 172.21.15.119:0/3772534148 learned_addr learned my addr 172.21.15.119:0/3772534148 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:37.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.482+0000 7f0e2f7fe700 1 -- 172.21.15.119:0/3772534148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e301072a0 con 0x7f0e30104840 2023-01-25T11:40:37.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.482+0000 7f0e2f7fe700 1 --2- 172.21.15.119:0/3772534148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e30106c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0e240098a0 tx=0x7f0e24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d98438b789448d5 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:37.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.482+0000 7f0e2e7fc700 1 -- 172.21.15.119:0/3772534148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24004160 con 0x7f0e30104840 2023-01-25T11:40:37.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e2e7fc700 1 -- 172.21.15.119:0/3772534148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f0e240036a0 con 0x7f0e30104840 2023-01-25T11:40:37.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e2e7fc700 1 -- 172.21.15.119:0/3772534148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24011870 con 0x7f0e30104840 2023-01-25T11:40:37.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e36261700 1 -- 172.21.15.119:0/3772534148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 msgr2=0x7f0e30106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:37.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e36261700 1 --2- 172.21.15.119:0/3772534148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e30106c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0e240098a0 tx=0x7f0e24009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:37.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e36261700 1 -- 172.21.15.119:0/3772534148 shutdown_connections 2023-01-25T11:40:37.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e36261700 1 --2- 172.21.15.119:0/3772534148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e30106c20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:37.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e36261700 1 -- 172.21.15.119:0/3772534148 >> 172.21.15.119:0/3772534148 conn(0x7f0e300fe220 msgr2=0x7f0e30100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:37.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e36261700 1 -- 172.21.15.119:0/3772534148 shutdown_connections 2023-01-25T11:40:37.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.483+0000 7f0e36261700 1 -- 172.21.15.119:0/3772534148 wait complete. 2023-01-25T11:40:37.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.484+0000 7f0e36261700 1 Processor -- start 2023-01-25T11:40:37.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.484+0000 7f0e36261700 1 -- start start 2023-01-25T11:40:37.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.484+0000 7f0e36261700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e301041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:37.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.484+0000 7f0e36261700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e301046f0 con 0x7f0e30104840 2023-01-25T11:40:37.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.484+0000 7f0e2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e301041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:37.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.484+0000 7f0e2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e301041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39152/0 (socket says 172.21.15.119:39152) 2023-01-25T11:40:37.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.484+0000 7f0e2f7fe700 1 -- 172.21.15.119:0/744022744 learned_addr learned my addr 172.21.15.119:0/744022744 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:37.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2f7fe700 1 -- 172.21.15.119:0/744022744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e24009550 con 0x7f0e30104840 2023-01-25T11:40:37.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2f7fe700 1 --2- 172.21.15.119:0/744022744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e301041b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0e240119c0 tx=0x7f0e24011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:37.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 1 -- 172.21.15.119:0/744022744 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24004110 con 0x7f0e30104840 2023-01-25T11:40:37.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e30108110 con 0x7f0e30104840 2023-01-25T11:40:37.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 1 -- 172.21.15.119:0/744022744 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f0e2401b440 con 0x7f0e30104840 2023-01-25T11:40:37.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 1 -- 172.21.15.119:0/744022744 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0e24023400 con 0x7f0e30104840 2023-01-25T11:40:37.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e30107f80 con 0x7f0e30104840 2023-01-25T11:40:37.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 1 -- 172.21.15.119:0/744022744 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f0e24017020 con 0x7f0e30104840 2023-01-25T11:40:37.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-01-25T11:40:37.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 4 mgrc handle_mgr_map Got map version 1 2023-01-25T11:40:37.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:40:37.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:40:37.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.485+0000 7f0e2cff9700 1 -- 172.21.15.119:0/744022744 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0e2402cbc0 con 0x7f0e30104840 2023-01-25T11:40:37.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.486+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e301082a0 con 0x7f0e30104840 2023-01-25T11:40:37.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.488+0000 7f0e2cff9700 1 -- 172.21.15.119:0/744022744 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f0e2401b950 con 0x7f0e30104840 2023-01-25T11:40:37.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.555+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f0e30103050 con 0x7f0e30104840 2023-01-25T11:40:37.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e2cff9700 1 -- 172.21.15.119:0/744022744 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0e2401b5b0 con 0x7f0e30104840 2023-01-25T11:40:37.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e36261700 10 mgrc shutdown 2023-01-25T11:40:37.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 msgr2=0x7f0e301041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:37.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e36261700 1 --2- 172.21.15.119:0/744022744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e301041b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0e240119c0 tx=0x7f0e24011b30 comp rx=0 tx=0).stop 2023-01-25T11:40:37.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 shutdown_connections 2023-01-25T11:40:37.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e36261700 1 --2- 172.21.15.119:0/744022744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e30104840 0x7f0e301041b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:37.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 >> 172.21.15.119:0/744022744 conn(0x7f0e300fe220 msgr2=0x7f0e300feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:37.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.558+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 shutdown_connections 2023-01-25T11:40:37.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:37.559+0000 7f0e36261700 1 -- 172.21.15.119:0/744022744 wait complete. 2023-01-25T11:40:37.987 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote config to /etc/ceph/ceph.conf 2023-01-25T11:40:37.987 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-01-25T11:40:37.987 INFO:teuthology.orchestra.run.smithi119.stdout:Creating mgr... 2023-01-25T11:40:37.987 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 9283 ... 2023-01-25T11:40:37.988 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying port 8765 ... 2023-01-25T11:40:38.102 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mgr.a 2023-01-25T11:40:38.102 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mgr.a.service: Unit ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mgr.a.service not loaded. 2023-01-25T11:40:38.190 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c.target.wants/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@mgr.a.service → /etc/systemd/system/ceph-f64523de-9ca4-11ed-9e56-001a4aab830c@.service. 2023-01-25T11:40:38.390 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:38 smithi119 systemd[1]: Starting Ceph mgr.a for f64523de-9ca4-11ed-9e56-001a4aab830c... 2023-01-25T11:40:38.843 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:38 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/744022744' entity='client.admin' 2023-01-25T11:40:39.071 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-01-25T11:40:39.072 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-01-25T11:40:39.079 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-01-25T11:40:39.080 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-01-25T11:40:39.080 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-01-25T11:40:39.080 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to enable service . firewalld.service is not available 2023-01-25T11:40:39.088 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-01-25T11:40:39.088 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-01-25T11:40:39.096 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-01-25T11:40:39.096 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-01-25T11:40:39.096 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-01-25T11:40:39.097 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-01-25T11:40:39.097 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr to start... 2023-01-25T11:40:39.097 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr... 2023-01-25T11:40:39.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:38 smithi119 podman[102369]: 2023-01-25T11:40:39.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:39 smithi119 bash[102369]: bd390c4f4feddbe31fc47aa9cfb4657fdc1448493c913e22c44bebb5e792356c 2023-01-25T11:40:39.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:39 smithi119 systemd[1]: Started Ceph mgr.a for f64523de-9ca4-11ed-9e56-001a4aab830c. 2023-01-25T11:40:39.961 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:39 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/1907138982' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-01-25T11:40:40.141 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:40.142 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:40:40.142 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "f64523de-9ca4-11ed-9e56-001a4aab830c", 2023-01-25T11:40:40.142 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2023-01-25T11:40:40.142 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-01-25T11:40:40.142 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-01-25T11:40:40.143 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-01-25T11:40:40.143 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:40.143 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-01-25T11:40:40.143 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-01-25T11:40:40.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2023-01-25T11:40:40.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:40.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-01-25T11:40:40.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2023-01-25T11:40:40.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:40.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-01-25T11:40:40.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2023-01-25T11:40:40.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:40.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-01-25T11:40:40.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-01-25T11:40:40.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:40.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-01-25T11:40:40.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:40.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-01-25T11:40:40.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-01-25T11:40:40.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-01-25T11:40:40.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-01-25T11:40:40.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-01-25T11:40:40.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-01-25T11:40:40.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:40.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-01-25T11:40:40.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-01-25T11:40:40.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-01-25T11:40:40.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-01-25T11:40:40.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-01-25T11:40:40.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-01-25T11:40:40.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-01-25T11:40:40.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-01-25T11:40:40.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-01-25T11:40:40.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:40.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-01-25T11:40:40.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:40.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-01-25T11:40:40.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-01-25T11:40:40.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:40.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-01-25T11:40:40.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2023-01-25T11:40:40.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-01-25T11:40:40.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2023-01-25T11:40:40.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2023-01-25T11:40:40.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2023-01-25T11:40:40.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2023-01-25T11:40:40.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:40.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:40.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:40.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-01-25T11:40:40.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:40.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2023-01-25T11:40:32.230585+0000", 2023-01-25T11:40:40.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:40.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:40.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-01-25T11:40:40.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:40:40.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.579+0000 7f5b5a1c5700 1 Processor -- start 2023-01-25T11:40:40.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.579+0000 7f5b5a1c5700 1 -- start start 2023-01-25T11:40:40.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.579+0000 7f5b5a1c5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b541047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:40.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.579+0000 7f5b5a1c5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54104da0 con 0x7f5b541043c0 2023-01-25T11:40:40.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b541047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:40.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b541047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39186/0 (socket says 172.21.15.119:39186) 2023-01-25T11:40:40.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b537fe700 1 -- 172.21.15.119:0/777869794 learned_addr learned my addr 172.21.15.119:0/777869794 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:40.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b537fe700 1 -- 172.21.15.119:0/777869794 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b541028f0 con 0x7f5b541043c0 2023-01-25T11:40:40.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b537fe700 1 --2- 172.21.15.119:0/777869794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b541047d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b440098a0 tx=0x7f5b44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a5042786d52ce3 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:40.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b527fc700 1 -- 172.21.15.119:0/777869794 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b44004160 con 0x7f5b541043c0 2023-01-25T11:40:40.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b527fc700 1 -- 172.21.15.119:0/777869794 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f5b440036a0 con 0x7f5b541043c0 2023-01-25T11:40:40.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.580+0000 7f5b527fc700 1 -- 172.21.15.119:0/777869794 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b44011870 con 0x7f5b541043c0 2023-01-25T11:40:40.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.581+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/777869794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 msgr2=0x7f5b541047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:40.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.581+0000 7f5b5a1c5700 1 --2- 172.21.15.119:0/777869794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b541047d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f5b440098a0 tx=0x7f5b44009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:40.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.581+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/777869794 shutdown_connections 2023-01-25T11:40:40.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.581+0000 7f5b5a1c5700 1 --2- 172.21.15.119:0/777869794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b541047d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:40.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.581+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/777869794 >> 172.21.15.119:0/777869794 conn(0x7f5b540fe220 msgr2=0x7f5b54100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:40.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.581+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/777869794 shutdown_connections 2023-01-25T11:40:40.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.581+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/777869794 wait complete. 2023-01-25T11:40:40.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.582+0000 7f5b5a1c5700 1 Processor -- start 2023-01-25T11:40:40.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.582+0000 7f5b5a1c5700 1 -- start start 2023-01-25T11:40:40.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.582+0000 7f5b5a1c5700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b5418df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:40.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.582+0000 7f5b5a1c5700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5418e4a0 con 0x7f5b541043c0 2023-01-25T11:40:40.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.582+0000 7f5b537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b5418df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:40.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b5418df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39194/0 (socket says 172.21.15.119:39194) 2023-01-25T11:40:40.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b537fe700 1 -- 172.21.15.119:0/1907138982 learned_addr learned my addr 172.21.15.119:0/1907138982 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:40.167 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b537fe700 1 -- 172.21.15.119:0/1907138982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b44009550 con 0x7f5b541043c0 2023-01-25T11:40:40.167 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b537fe700 1 --2- 172.21.15.119:0/1907138982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b5418df60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5b44003f20 tx=0x7f5b44011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:40.167 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b50ff9700 1 -- 172.21.15.119:0/1907138982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b44004110 con 0x7f5b541043c0 2023-01-25T11:40:40.167 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b50ff9700 1 -- 172.21.15.119:0/1907138982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f5b4401b440 con 0x7f5b541043c0 2023-01-25T11:40:40.167 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b54107100 con 0x7f5b541043c0 2023-01-25T11:40:40.168 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b50ff9700 1 -- 172.21.15.119:0/1907138982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b440193d0 con 0x7f5b541043c0 2023-01-25T11:40:40.168 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.583+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541036a0 con 0x7f5b541043c0 2023-01-25T11:40:40.168 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.584+0000 7f5b50ff9700 1 -- 172.21.15.119:0/1907138982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5b4402a070 con 0x7f5b541043c0 2023-01-25T11:40:40.168 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.584+0000 7f5b50ff9700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-01-25T11:40:40.168 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.584+0000 7f5b50ff9700 4 mgrc handle_mgr_map Got map version 1 2023-01-25T11:40:40.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.584+0000 7f5b50ff9700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:40:40.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.584+0000 7f5b50ff9700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:40:40.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.584+0000 7f5b50ff9700 1 -- 172.21.15.119:0/1907138982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5b440238b0 con 0x7f5b541043c0 2023-01-25T11:40:40.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.584+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b40005110 con 0x7f5b541043c0 2023-01-25T11:40:40.169 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.586+0000 7f5b50ff9700 1 -- 172.21.15.119:0/1907138982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f5b44028060 con 0x7f5b541043c0 2023-01-25T11:40:40.170 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.658+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f5b40005ab0 con 0x7f5b541043c0 2023-01-25T11:40:40.170 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.658+0000 7f5b50ff9700 1 -- 172.21.15.119:0/1907138982 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f5b44023400 con 0x7f5b541043c0 2023-01-25T11:40:40.170 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 10 mgrc shutdown 2023-01-25T11:40:40.170 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 msgr2=0x7f5b5418df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:40.170 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 1 --2- 172.21.15.119:0/1907138982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b5418df60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f5b44003f20 tx=0x7f5b44011ae0 comp rx=0 tx=0).stop 2023-01-25T11:40:40.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 shutdown_connections 2023-01-25T11:40:40.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 1 --2- 172.21.15.119:0/1907138982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b541043c0 0x7f5b5418df60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:40.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 >> 172.21.15.119:0/1907138982 conn(0x7f5b540fe220 msgr2=0x7f5b54077120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:40.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 shutdown_connections 2023-01-25T11:40:40.171 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:39.659+0000 7f5b5a1c5700 1 -- 172.21.15.119:0/1907138982 wait complete. 2023-01-25T11:40:40.172 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (1/15)... 2023-01-25T11:40:40.390 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:39 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:39.958+0000 7f5e5155c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T11:40:40.791 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:40 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:40.530+0000 7f5e5155c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T11:40:40.791 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:40 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:40.617+0000 7f5e5155c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T11:40:41.890 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:40 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:40.789+0000 7f5e5155c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T11:40:41.891 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:41 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:41.580+0000 7f5e5155c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T11:40:41.891 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:41 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:41.758+0000 7f5e5155c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T11:40:42.282 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:42 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:42.023+0000 7f5e5155c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T11:40:42.283 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:42 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:42.108+0000 7f5e5155c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T11:40:42.580 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:42 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:42.280+0000 7f5e5155c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T11:40:42.890 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:42 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:42.578+0000 7f5e5155c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T11:40:42.890 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:42 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:42.670+0000 7f5e5155c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T11:40:43.085 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:43.087 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:40:43.087 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "f64523de-9ca4-11ed-9e56-001a4aab830c", 2023-01-25T11:40:43.087 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2023-01-25T11:40:43.088 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-01-25T11:40:43.088 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-01-25T11:40:43.088 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-01-25T11:40:43.088 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:43.088 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-01-25T11:40:43.089 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-01-25T11:40:43.089 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2023-01-25T11:40:43.089 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:43.089 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-01-25T11:40:43.089 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2023-01-25T11:40:43.089 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:43.090 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2023-01-25T11:40:43.090 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2023-01-25T11:40:43.090 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:43.090 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-01-25T11:40:43.090 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-01-25T11:40:43.091 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:43.091 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-01-25T11:40:43.091 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:43.091 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-01-25T11:40:43.091 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-01-25T11:40:43.091 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-01-25T11:40:43.099 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-01-25T11:40:43.099 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-01-25T11:40:43.099 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-01-25T11:40:43.099 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:43.100 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-01-25T11:40:43.100 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-01-25T11:40:43.100 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-01-25T11:40:43.100 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-01-25T11:40:43.100 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-01-25T11:40:43.101 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-01-25T11:40:43.101 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-01-25T11:40:43.101 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-01-25T11:40:43.101 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-01-25T11:40:43.101 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:43.102 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-01-25T11:40:43.102 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:43.102 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-01-25T11:40:43.102 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-01-25T11:40:43.102 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:43.102 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-01-25T11:40:43.103 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2023-01-25T11:40:43.103 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-01-25T11:40:43.103 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2023-01-25T11:40:43.103 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2023-01-25T11:40:43.103 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2023-01-25T11:40:43.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2023-01-25T11:40:43.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:43.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:43.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:43.104 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-01-25T11:40:43.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:43.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2023-01-25T11:40:32.230585+0000", 2023-01-25T11:40:43.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:43.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:43.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-01-25T11:40:43.105 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:40:43.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a7ad5b700 1 Processor -- start 2023-01-25T11:40:43.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a7ad5b700 1 -- start start 2023-01-25T11:40:43.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a7ad5b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:43.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a7ad5b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a74107160 con 0x7f9a74104840 2023-01-25T11:40:43.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a78af7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:43.106 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a78af7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39204/0 (socket says 172.21.15.119:39204) 2023-01-25T11:40:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a78af7700 1 -- 172.21.15.119:0/1901170103 learned_addr learned my addr 172.21.15.119:0/1901170103 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a78af7700 1 -- 172.21.15.119:0/1901170103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a741072a0 con 0x7f9a74104840 2023-01-25T11:40:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.590+0000 7f9a78af7700 1 --2- 172.21.15.119:0/1901170103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74106c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009e80 tx=0x7f9a7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5705fc633b539b server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a6b7fe700 1 -- 172.21.15.119:0/1901170103 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a7000e070 con 0x7f9a74104840 2023-01-25T11:40:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a6b7fe700 1 -- 172.21.15.119:0/1901170103 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f9a70003e70 con 0x7f9a74104840 2023-01-25T11:40:43.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a6b7fe700 1 -- 172.21.15.119:0/1901170103 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a70004300 con 0x7f9a74104840 2023-01-25T11:40:43.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/1901170103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 msgr2=0x7f9a74106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:43.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a7ad5b700 1 --2- 172.21.15.119:0/1901170103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74106c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009e80 tx=0x7f9a7000b330 comp rx=0 tx=0).stop 2023-01-25T11:40:43.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/1901170103 shutdown_connections 2023-01-25T11:40:43.108 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a7ad5b700 1 --2- 172.21.15.119:0/1901170103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74106c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:43.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/1901170103 >> 172.21.15.119:0/1901170103 conn(0x7f9a740fe220 msgr2=0x7f9a74100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:43.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/1901170103 shutdown_connections 2023-01-25T11:40:43.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.591+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/1901170103 wait complete. 2023-01-25T11:40:43.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a7ad5b700 1 Processor -- start 2023-01-25T11:40:43.109 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a7ad5b700 1 -- start start 2023-01-25T11:40:43.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a7ad5b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:43.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a7ad5b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a74197ff0 con 0x7f9a74104840 2023-01-25T11:40:43.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a78af7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:43.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a78af7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39210/0 (socket says 172.21.15.119:39210) 2023-01-25T11:40:43.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a78af7700 1 -- 172.21.15.119:0/3680075861 learned_addr learned my addr 172.21.15.119:0/3680075861 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:43.110 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.592+0000 7f9a78af7700 1 -- 172.21.15.119:0/3680075861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009b30 con 0x7f9a74104840 2023-01-25T11:40:43.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a78af7700 1 --2- 172.21.15.119:0/3680075861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74197ab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a70004000 tx=0x7f9a7000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:43.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 1 -- 172.21.15.119:0/3680075861 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a7000e040 con 0x7f9a74104840 2023-01-25T11:40:43.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 1 -- 172.21.15.119:0/3680075861 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f9a70004820 con 0x7f9a74104840 2023-01-25T11:40:43.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a74108110 con 0x7f9a74104840 2023-01-25T11:40:43.111 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 1 -- 172.21.15.119:0/3680075861 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a7001c590 con 0x7f9a74104840 2023-01-25T11:40:43.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a74107f80 con 0x7f9a74104840 2023-01-25T11:40:43.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 1 -- 172.21.15.119:0/3680075861 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9a7002c070 con 0x7f9a74104840 2023-01-25T11:40:43.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-01-25T11:40:43.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 4 mgrc handle_mgr_map Got map version 1 2023-01-25T11:40:43.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:40:43.112 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:40:43.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.593+0000 7f9a69ffb700 1 -- 172.21.15.119:0/3680075861 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9a70025b80 con 0x7f9a74104840 2023-01-25T11:40:43.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.594+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a741082a0 con 0x7f9a74104840 2023-01-25T11:40:43.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.596+0000 7f9a69ffb700 1 -- 172.21.15.119:0/3680075861 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f9a7002a060 con 0x7f9a74104840 2023-01-25T11:40:43.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.670+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9a74198130 con 0x7f9a74104840 2023-01-25T11:40:43.113 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.670+0000 7f9a69ffb700 1 -- 172.21.15.119:0/3680075861 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9a70020070 con 0x7f9a74104840 2023-01-25T11:40:43.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 10 mgrc shutdown 2023-01-25T11:40:43.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 msgr2=0x7f9a74197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:43.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 1 --2- 172.21.15.119:0/3680075861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74197ab0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a70004000 tx=0x7f9a7000be20 comp rx=0 tx=0).stop 2023-01-25T11:40:43.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 shutdown_connections 2023-01-25T11:40:43.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 1 --2- 172.21.15.119:0/3680075861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a74104840 0x7f9a74197ab0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:43.114 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 >> 172.21.15.119:0/3680075861 conn(0x7f9a740fe220 msgr2=0x7f9a740fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:43.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 shutdown_connections 2023-01-25T11:40:43.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:42.671+0000 7f9a7ad5b700 1 -- 172.21.15.119:0/3680075861 wait complete. 2023-01-25T11:40:43.115 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (2/15)... 2023-01-25T11:40:43.285 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:43 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/3680075861' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-01-25T11:40:43.285 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:42 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:42.976+0000 7f5e5155c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T11:40:43.285 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:43 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:43.109+0000 7f5e5155c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T11:40:43.640 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:43 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:43.282+0000 7f5e5155c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T11:40:43.641 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:43 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:43.375+0000 7f5e5155c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T11:40:43.641 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:43 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:43.467+0000 7f5e5155c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T11:40:44.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:43 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:43.727+0000 7f5e5155c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T11:40:44.699 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:44 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:44.418+0000 7f5e5155c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T11:40:44.700 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:44 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:44.513+0000 7f5e5155c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T11:40:44.700 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:44 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:44.601+0000 7f5e5155c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T11:40:45.383 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:44 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:44.697+0000 7f5e5155c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T11:40:45.383 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:45.116+0000 7f5e5155c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T11:40:45.383 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:45.207+0000 7f5e5155c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T11:40:45.640 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:45.394+0000 7f5e5155c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T11:40:45.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: Activating manager daemon a 2023-01-25T11:40:45.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: mgrmap e2: a(active, starting, since 0.00282401s) 2023-01-25T11:40:45.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-01-25T11:40:45.641 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-01-25T11:40:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-01-25T11:40:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-01-25T11:40:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' cmd=[{"prefix": "mgr metadata", "who": "a", "id": "a"}]: dispatch 2023-01-25T11:40:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: Manager daemon a is now available 2023-01-25T11:40:45.642 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/a/mirror_snapshot_schedule"}]: dispatch 2023-01-25T11:40:45.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/a/trash_purge_schedule"}]: dispatch 2023-01-25T11:40:45.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' 2023-01-25T11:40:45.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' 2023-01-25T11:40:45.643 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:45 smithi119 ceph-mon[102162]: from='mgr.14100 172.21.15.119:0/1283283407' entity='mgr.a' 2023-01-25T11:40:46.039 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:46.039 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:40:46.040 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "f64523de-9ca4-11ed-9e56-001a4aab830c", 2023-01-25T11:40:46.041 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2023-01-25T11:40:46.041 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-01-25T11:40:46.041 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-01-25T11:40:46.041 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-01-25T11:40:46.041 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:46.042 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-01-25T11:40:46.042 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-01-25T11:40:46.042 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2023-01-25T11:40:46.042 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:46.043 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-01-25T11:40:46.043 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2023-01-25T11:40:46.043 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:46.043 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-01-25T11:40:46.043 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2023-01-25T11:40:46.044 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:46.044 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-01-25T11:40:46.044 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-01-25T11:40:46.044 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:46.044 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-01-25T11:40:46.045 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:46.045 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-01-25T11:40:46.045 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-01-25T11:40:46.045 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-01-25T11:40:46.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-01-25T11:40:46.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-01-25T11:40:46.054 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-01-25T11:40:46.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:46.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-01-25T11:40:46.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-01-25T11:40:46.055 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-01-25T11:40:46.056 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-01-25T11:40:46.056 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-01-25T11:40:46.056 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-01-25T11:40:46.056 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-01-25T11:40:46.056 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-01-25T11:40:46.057 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-01-25T11:40:46.057 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:46.057 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-01-25T11:40:46.057 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:46.058 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-01-25T11:40:46.058 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-01-25T11:40:46.058 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:46.058 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-01-25T11:40:46.058 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": false, 2023-01-25T11:40:46.059 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-01-25T11:40:46.059 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2023-01-25T11:40:46.059 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2023-01-25T11:40:46.059 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2023-01-25T11:40:46.060 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2023-01-25T11:40:46.060 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:46.060 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:46.060 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:46.061 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-01-25T11:40:46.061 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:46.061 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2023-01-25T11:40:32.230585+0000", 2023-01-25T11:40:46.061 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:46.062 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:46.062 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-01-25T11:40:46.062 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:40:46.062 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.501+0000 7f272d183700 1 Processor -- start 2023-01-25T11:40:46.062 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.501+0000 7f272d183700 1 -- start start 2023-01-25T11:40:46.063 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.501+0000 7f272d183700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:46.063 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.501+0000 7f272d183700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728077f50 con 0x7f27281026b0 2023-01-25T11:40:46.063 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2726d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:46.063 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2726d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39286/0 (socket says 172.21.15.119:39286) 2023-01-25T11:40:46.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2726d9d700 1 -- 172.21.15.119:0/1548699040 learned_addr learned my addr 172.21.15.119:0/1548699040 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:46.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2726d9d700 1 -- 172.21.15.119:0/1548699040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2728103000 con 0x7f27281026b0 2023-01-25T11:40:46.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2726d9d700 1 --2- 172.21.15.119:0/1548699040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728102ac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27180098a0 tx=0x7f2718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8189a11cdf78fe server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:46.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2725d9b700 1 -- 172.21.15.119:0/1548699040 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2718004160 con 0x7f27281026b0 2023-01-25T11:40:46.064 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2725d9b700 1 -- 172.21.15.119:0/1548699040 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f27180036a0 con 0x7f27281026b0 2023-01-25T11:40:46.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.502+0000 7f2725d9b700 1 -- 172.21.15.119:0/1548699040 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2718011870 con 0x7f27281026b0 2023-01-25T11:40:46.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 -- 172.21.15.119:0/1548699040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 msgr2=0x7f2728102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:46.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 --2- 172.21.15.119:0/1548699040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728102ac0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f27180098a0 tx=0x7f2718009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:46.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 -- 172.21.15.119:0/1548699040 shutdown_connections 2023-01-25T11:40:46.065 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 --2- 172.21.15.119:0/1548699040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728102ac0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:46.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 -- 172.21.15.119:0/1548699040 >> 172.21.15.119:0/1548699040 conn(0x7f27280fe220 msgr2=0x7f2728100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:46.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 -- 172.21.15.119:0/1548699040 shutdown_connections 2023-01-25T11:40:46.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 -- 172.21.15.119:0/1548699040 wait complete. 2023-01-25T11:40:46.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.503+0000 7f272d183700 1 Processor -- start 2023-01-25T11:40:46.066 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f272d183700 1 -- start start 2023-01-25T11:40:46.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f272d183700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:46.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f272d183700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272819a2a0 con 0x7f27281026b0 2023-01-25T11:40:46.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f2726d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:46.067 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f2726d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39292/0 (socket says 172.21.15.119:39292) 2023-01-25T11:40:46.068 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f2726d9d700 1 -- 172.21.15.119:0/2747639765 learned_addr learned my addr 172.21.15.119:0/2747639765 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:46.068 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f2726d9d700 1 -- 172.21.15.119:0/2747639765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2718009550 con 0x7f27281026b0 2023-01-25T11:40:46.068 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f2726d9d700 1 --2- 172.21.15.119:0/2747639765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728199d60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f27180119c0 tx=0x7f2718011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:46.068 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.504+0000 7f270ffff700 1 -- 172.21.15.119:0/2747639765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2718004110 con 0x7f27281026b0 2023-01-25T11:40:46.068 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.505+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2728103ee0 con 0x7f27281026b0 2023-01-25T11:40:46.069 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.505+0000 7f270ffff700 1 -- 172.21.15.119:0/2747639765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f271801b440 con 0x7f27281026b0 2023-01-25T11:40:46.069 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.505+0000 7f270ffff700 1 -- 172.21.15.119:0/2747639765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2718023400 con 0x7f27281026b0 2023-01-25T11:40:46.069 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.505+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2728103d50 con 0x7f27281026b0 2023-01-25T11:40:46.069 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.505+0000 7f270ffff700 1 -- 172.21.15.119:0/2747639765 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 2) v1 ==== 43308+0+0 (secure 0 0 0) 0x7f2718017020 con 0x7f27281026b0 2023-01-25T11:40:46.070 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.505+0000 7f270ffff700 20 mgrc handle_mgr_map mgrmap(e 2) v1 2023-01-25T11:40:46.070 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.505+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2728104070 con 0x7f27281026b0 2023-01-25T11:40:46.070 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.506+0000 7f270ffff700 4 mgrc handle_mgr_map Got map version 2 2023-01-25T11:40:46.070 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.506+0000 7f270ffff700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:40:46.071 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.506+0000 7f270ffff700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:40:46.071 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.506+0000 7f270ffff700 1 -- 172.21.15.119:0/2747639765 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2718049bb0 con 0x7f27281026b0 2023-01-25T11:40:46.071 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.508+0000 7f270ffff700 1 -- 172.21.15.119:0/2747639765 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71752 (secure 0 0 0) 0x7f271802c3f0 con 0x7f27281026b0 2023-01-25T11:40:46.071 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.579+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f2728193630 con 0x7f27281026b0 2023-01-25T11:40:46.071 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.579+0000 7f270ffff700 1 -- 172.21.15.119:0/2747639765 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f271800b3e0 con 0x7f27281026b0 2023-01-25T11:40:46.072 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 10 mgrc shutdown 2023-01-25T11:40:46.072 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 msgr2=0x7f2728199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:46.072 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 1 --2- 172.21.15.119:0/2747639765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728199d60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f27180119c0 tx=0x7f2718011b30 comp rx=0 tx=0).stop 2023-01-25T11:40:46.072 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 shutdown_connections 2023-01-25T11:40:46.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 1 --2- 172.21.15.119:0/2747639765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27281026b0 0x7f2728199d60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:46.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 >> 172.21.15.119:0/2747639765 conn(0x7f27280fe220 msgr2=0x7f272818e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:46.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 shutdown_connections 2023-01-25T11:40:46.073 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:45.580+0000 7f272d183700 1 -- 172.21.15.119:0/2747639765 wait complete. 2023-01-25T11:40:46.073 INFO:teuthology.orchestra.run.smithi119.stdout:mgr not available, waiting (3/15)... 2023-01-25T11:40:46.890 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:46 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/2747639765' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-01-25T11:40:46.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:46 smithi119 ceph-mon[102162]: mgrmap e3: a(active, since 1.00595s) 2023-01-25T11:40:49.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:49 smithi119 ceph-mon[102162]: mgrmap e4: a(active, since 2s) 2023-01-25T11:40:51.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:51.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:40:51.593 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsid": "f64523de-9ca4-11ed-9e56-001a4aab830c", 2023-01-25T11:40:51.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "health": { 2023-01-25T11:40:51.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-01-25T11:40:51.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-01-25T11:40:51.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-01-25T11:40:51.594 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:51.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-01-25T11:40:51.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-01-25T11:40:51.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 0 2023-01-25T11:40:51.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:51.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-01-25T11:40:51.595 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "a" 2023-01-25T11:40:51.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:51.596 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2023-01-25T11:40:51.596 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:51 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/3752579652' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-01-25T11:40:51.597 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "monmap": { 2023-01-25T11:40:51.598 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:51.598 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-01-25T11:40:51.598 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-01-25T11:40:51.598 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:51.598 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-01-25T11:40:51.599 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:51.599 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-01-25T11:40:51.599 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-01-25T11:40:51.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-01-25T11:40:51.607 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-01-25T11:40:51.608 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-01-25T11:40:51.608 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-01-25T11:40:51.608 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:51.608 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-01-25T11:40:51.608 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-01-25T11:40:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-01-25T11:40:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-01-25T11:40:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-01-25T11:40:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-01-25T11:40:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-01-25T11:40:51.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-01-25T11:40:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-01-25T11:40:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-01-25T11:40:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:51.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-01-25T11:40:51.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-01-25T11:40:51.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:51.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-01-25T11:40:51.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2023-01-25T11:40:51.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-01-25T11:40:51.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modules": [ 2023-01-25T11:40:51.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "iostat", 2023-01-25T11:40:51.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "nfs", 2023-01-25T11:40:51.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "restful" 2023-01-25T11:40:51.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ], 2023-01-25T11:40:51.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:51.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:51.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-01-25T11:40:51.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-01-25T11:40:51.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "modified": "2023-01-25T11:40:32.230585+0000", 2023-01-25T11:40:51.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "services": {} 2023-01-25T11:40:51.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout }, 2023-01-25T11:40:51.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-01-25T11:40:51.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:40:51.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.857+0000 7f3b364ec700 1 Processor -- start 2023-01-25T11:40:51.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.857+0000 7f3b364ec700 1 -- start start 2023-01-25T11:40:51.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.857+0000 7f3b364ec700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30104330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:51.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.857+0000 7f3b364ec700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30104870 con 0x7f3b30103f20 2023-01-25T11:40:51.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.857+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30104330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:51.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.857+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30104330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35954/0 (socket says 172.21.15.119:35954) 2023-01-25T11:40:51.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.857+0000 7f3b2ffff700 1 -- 172.21.15.119:0/4016927146 learned_addr learned my addr 172.21.15.119:0/4016927146 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:51.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.858+0000 7f3b2ffff700 1 -- 172.21.15.119:0/4016927146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b301049b0 con 0x7f3b30103f20 2023-01-25T11:40:51.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.858+0000 7f3b2ffff700 1 --2- 172.21.15.119:0/4016927146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30104330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3b200098a0 tx=0x7f3b20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584db9f1cc9ff721 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:51.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.858+0000 7f3b2f7fe700 1 -- 172.21.15.119:0/4016927146 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b20004160 con 0x7f3b30103f20 2023-01-25T11:40:51.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.858+0000 7f3b2f7fe700 1 -- 172.21.15.119:0/4016927146 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f3b200036a0 con 0x7f3b30103f20 2023-01-25T11:40:51.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.858+0000 7f3b2f7fe700 1 -- 172.21.15.119:0/4016927146 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b20011870 con 0x7f3b30103f20 2023-01-25T11:40:51.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 -- 172.21.15.119:0/4016927146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 msgr2=0x7f3b30104330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:51.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 --2- 172.21.15.119:0/4016927146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30104330 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3b200098a0 tx=0x7f3b20009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:51.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 -- 172.21.15.119:0/4016927146 shutdown_connections 2023-01-25T11:40:51.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 --2- 172.21.15.119:0/4016927146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30104330 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:51.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 -- 172.21.15.119:0/4016927146 >> 172.21.15.119:0/4016927146 conn(0x7f3b300ff4a0 msgr2=0x7f3b301018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:51.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 -- 172.21.15.119:0/4016927146 shutdown_connections 2023-01-25T11:40:51.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 -- 172.21.15.119:0/4016927146 wait complete. 2023-01-25T11:40:51.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 Processor -- start 2023-01-25T11:40:51.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.859+0000 7f3b364ec700 1 -- start start 2023-01-25T11:40:51.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b364ec700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:51.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b364ec700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b30197ff0 con 0x7f3b30103f20 2023-01-25T11:40:51.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:51.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35960/0 (socket says 172.21.15.119:35960) 2023-01-25T11:40:51.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2ffff700 1 -- 172.21.15.119:0/3752579652 learned_addr learned my addr 172.21.15.119:0/3752579652 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:51.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2ffff700 1 -- 172.21.15.119:0/3752579652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b20009550 con 0x7f3b30103f20 2023-01-25T11:40:51.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2ffff700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30197ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3b200119a0 tx=0x7f3b20011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:51.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2dffb700 1 -- 172.21.15.119:0/3752579652 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b20004110 con 0x7f3b30103f20 2023-01-25T11:40:51.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2dffb700 1 -- 172.21.15.119:0/3752579652 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f3b2001b440 con 0x7f3b30103f20 2023-01-25T11:40:51.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b30105e10 con 0x7f3b30103f20 2023-01-25T11:40:51.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b2dffb700 1 -- 172.21.15.119:0/3752579652 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b200193d0 con 0x7f3b30103f20 2023-01-25T11:40:51.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.860+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b30105c80 con 0x7f3b30103f20 2023-01-25T11:40:51.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b2dffb700 1 -- 172.21.15.119:0/3752579652 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 43498+0+0 (secure 0 0 0) 0x7f3b2002a070 con 0x7f3b30103f20 2023-01-25T11:40:51.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b2dffb700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2023-01-25T11:40:51.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b301082a0 con 0x7f3b30103f20 2023-01-25T11:40:51.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b2dffb700 4 mgrc handle_mgr_map Got map version 4 2023-01-25T11:40:51.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b2dffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:51.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b2dffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:51.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b2dffb700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f3b1c036700 0x7f3b1c038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:51.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.861+0000 7f3b2dffb700 1 -- 172.21.15.119:0/3752579652 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3b20049a60 con 0x7f3b30103f20 2023-01-25T11:40:51.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.862+0000 7f3b27fff700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f3b1c036700 0x7f3b1c038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:51.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.862+0000 7f3b27fff700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f3b1c036700 0x7f3b1c038bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006fd0 tx=0x7f3b18006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:51.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:50.866+0000 7f3b2dffb700 1 -- 172.21.15.119:0/3752579652 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f3b2002b940 con 0x7f3b30103f20 2023-01-25T11:40:51.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.129+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3b30198630 con 0x7f3b30103f20 2023-01-25T11:40:51.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.129+0000 7f3b2dffb700 1 -- 172.21.15.119:0/3752579652 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f3b2002b350 con 0x7f3b30103f20 2023-01-25T11:40:51.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.131+0000 7f3b364ec700 10 mgrc shutdown 2023-01-25T11:40:51.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.131+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f3b1c036700 msgr2=0x7f3b1c038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:51.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.131+0000 7f3b364ec700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f3b1c036700 0x7f3b1c038bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006fd0 tx=0x7f3b18006e40 comp rx=0 tx=0).stop 2023-01-25T11:40:51.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.131+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 msgr2=0x7f3b30197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:51.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.131+0000 7f3b364ec700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30197ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3b200119a0 tx=0x7f3b20011b30 comp rx=0 tx=0).stop 2023-01-25T11:40:51.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.132+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 shutdown_connections 2023-01-25T11:40:51.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.132+0000 7f3b364ec700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f3b1c036700 0x7f3b1c038bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:51.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.132+0000 7f3b364ec700 1 --2- 172.21.15.119:0/3752579652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b30103f20 0x7f3b30197ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:51.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.132+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 >> 172.21.15.119:0/3752579652 conn(0x7f3b300ff4a0 msgr2=0x7f3b301001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:51.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.132+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 shutdown_connections 2023-01-25T11:40:51.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.132+0000 7f3b364ec700 1 -- 172.21.15.119:0/3752579652 wait complete. 2023-01-25T11:40:51.624 INFO:teuthology.orchestra.run.smithi119.stdout:mgr is available 2023-01-25T11:40:52.640 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [global] 2023-01-25T11:40:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout fsid = f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:40:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-01-25T11:40:52.641 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-01-25T11:40:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-01-25T11:40:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-01-25T11:40:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [mgr] 2023-01-25T11:40:52.642 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-01-25T11:40:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 2023-01-25T11:40:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout [osd] 2023-01-25T11:40:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-01-25T11:40:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-01-25T11:40:52.643 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-01-25T11:40:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-01-25T11:40:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-01-25T11:40:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.974+0000 7fa98df5a700 1 Processor -- start 2023-01-25T11:40:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.974+0000 7fa98df5a700 1 -- start start 2023-01-25T11:40:52.644 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.974+0000 7fa98df5a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa9881058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.974+0000 7fa98df5a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa988105e20 con 0x7fa9881054d0 2023-01-25T11:40:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.974+0000 7fa9877fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa9881058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.974+0000 7fa9877fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa9881058e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35964/0 (socket says 172.21.15.119:35964) 2023-01-25T11:40:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.974+0000 7fa9877fe700 1 -- 172.21.15.119:0/285441062 learned_addr learned my addr 172.21.15.119:0/285441062 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:52.645 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.975+0000 7fa9877fe700 1 -- 172.21.15.119:0/285441062 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa988105f60 con 0x7fa9881054d0 2023-01-25T11:40:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.975+0000 7fa9877fe700 1 --2- 172.21.15.119:0/285441062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa9881058e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa9700098a0 tx=0x7fa970009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f39fb48ea592e8b9 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.975+0000 7fa9867fc700 1 -- 172.21.15.119:0/285441062 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa970004160 con 0x7fa9881054d0 2023-01-25T11:40:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.975+0000 7fa9867fc700 1 -- 172.21.15.119:0/285441062 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fa9700036a0 con 0x7fa9881054d0 2023-01-25T11:40:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.975+0000 7fa9867fc700 1 -- 172.21.15.119:0/285441062 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa970011870 con 0x7fa9881054d0 2023-01-25T11:40:52.646 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.975+0000 7fa98df5a700 1 -- 172.21.15.119:0/285441062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 msgr2=0x7fa9881058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.975+0000 7fa98df5a700 1 --2- 172.21.15.119:0/285441062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa9881058e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa9700098a0 tx=0x7fa970009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 -- 172.21.15.119:0/285441062 shutdown_connections 2023-01-25T11:40:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 --2- 172.21.15.119:0/285441062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa9881058e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 -- 172.21.15.119:0/285441062 >> 172.21.15.119:0/285441062 conn(0x7fa9880fdaf0 msgr2=0x7fa9880fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:52.647 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 -- 172.21.15.119:0/285441062 shutdown_connections 2023-01-25T11:40:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 -- 172.21.15.119:0/285441062 wait complete. 2023-01-25T11:40:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 Processor -- start 2023-01-25T11:40:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 -- start start 2023-01-25T11:40:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa988197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:52.648 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.976+0000 7fa98df5a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa988197f90 con 0x7fa9881054d0 2023-01-25T11:40:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa9877fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa988197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa9877fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa988197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35980/0 (socket says 172.21.15.119:35980) 2023-01-25T11:40:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa9877fe700 1 -- 172.21.15.119:0/2381541782 learned_addr learned my addr 172.21.15.119:0/2381541782 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa9877fe700 1 -- 172.21.15.119:0/2381541782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa970009550 con 0x7fa9881054d0 2023-01-25T11:40:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa9877fe700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa988197a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa9700119a0 tx=0x7fa970011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:52.649 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa984ff9700 1 -- 172.21.15.119:0/2381541782 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa970004110 con 0x7fa9881054d0 2023-01-25T11:40:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa984ff9700 1 -- 172.21.15.119:0/2381541782 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fa97001b440 con 0x7fa9881054d0 2023-01-25T11:40:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa988102b00 con 0x7fa9881054d0 2023-01-25T11:40:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa984ff9700 1 -- 172.21.15.119:0/2381541782 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9700193d0 con 0x7fa9881054d0 2023-01-25T11:40:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.977+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa988102970 con 0x7fa9881054d0 2023-01-25T11:40:52.650 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.978+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa988102c90 con 0x7fa9881054d0 2023-01-25T11:40:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.978+0000 7fa984ff9700 1 -- 172.21.15.119:0/2381541782 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 43498+0+0 (secure 0 0 0) 0x7fa97002a070 con 0x7fa9881054d0 2023-01-25T11:40:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.978+0000 7fa984ff9700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2023-01-25T11:40:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.978+0000 7fa984ff9700 4 mgrc handle_mgr_map Got map version 4 2023-01-25T11:40:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.979+0000 7fa984ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:52.651 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.979+0000 7fa984ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.979+0000 7fa984ff9700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fa974036700 0x7fa974038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.979+0000 7fa984ff9700 1 -- 172.21.15.119:0/2381541782 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa9700499f0 con 0x7fa9881054d0 2023-01-25T11:40:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.979+0000 7fa986ffd700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fa974036700 0x7fa974038bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.979+0000 7fa986ffd700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fa974036700 0x7fa974038bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa978006fd0 tx=0x7fa978006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:52.652 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:51.983+0000 7fa984ff9700 1 -- 172.21.15.119:0/2381541782 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fa97002b530 con 0x7fa9881054d0 2023-01-25T11:40:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.167+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa988061b00 con 0x7fa9881054d0 2023-01-25T11:40:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.168+0000 7fa984ff9700 1 -- 172.21.15.119:0/2381541782 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fa97002b350 con 0x7fa9881054d0 2023-01-25T11:40:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 10 mgrc shutdown 2023-01-25T11:40:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fa974036700 msgr2=0x7fa974038bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fa974036700 0x7fa974038bb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa978006fd0 tx=0x7fa978006e40 comp rx=0 tx=0).stop 2023-01-25T11:40:52.653 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 msgr2=0x7fa988197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa988197a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa9700119a0 tx=0x7fa970011b30 comp rx=0 tx=0).stop 2023-01-25T11:40:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 shutdown_connections 2023-01-25T11:40:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fa974036700 0x7fa974038bb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 --2- 172.21.15.119:0/2381541782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9881054d0 0x7fa988197a50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 >> 172.21.15.119:0/2381541782 conn(0x7fa9880fdaf0 msgr2=0x7fa9880fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:52.654 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 shutdown_connections 2023-01-25T11:40:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:52.170+0000 7fa98df5a700 1 -- 172.21.15.119:0/2381541782 wait complete. 2023-01-25T11:40:52.655 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling cephadm module... 2023-01-25T11:40:53.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:53 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/2381541782' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-01-25T11:40:54.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:54 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/2587617274' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-01-25T11:40:54.391 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:54 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: ignoring --setuser ceph since I am not root 2023-01-25T11:40:54.391 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:54 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: ignoring --setgroup ceph since I am not root 2023-01-25T11:40:54.480 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.008+0000 7fb0427a7700 1 Processor -- start 2023-01-25T11:40:54.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.008+0000 7fb0427a7700 1 -- start start 2023-01-25T11:40:54.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.008+0000 7fb0427a7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c106980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:54.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.008+0000 7fb0427a7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c106ec0 con 0x7fb03c077cc0 2023-01-25T11:40:54.481 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.008+0000 7fb03bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c106980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:54.482 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.008+0000 7fb03bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c106980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35996/0 (socket says 172.21.15.119:35996) 2023-01-25T11:40:54.482 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.009+0000 7fb03bfff700 1 -- 172.21.15.119:0/2529848776 learned_addr learned my addr 172.21.15.119:0/2529848776 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:54.482 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.009+0000 7fb03bfff700 1 -- 172.21.15.119:0/2529848776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c107000 con 0x7fb03c077cc0 2023-01-25T11:40:54.482 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.009+0000 7fb03bfff700 1 --2- 172.21.15.119:0/2529848776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c106980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098a0 tx=0x7fb02c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5911a2dbc5ae953 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:54.482 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.009+0000 7fb03b7fe700 1 -- 172.21.15.119:0/2529848776 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb02c004160 con 0x7fb03c077cc0 2023-01-25T11:40:54.482 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.009+0000 7fb03b7fe700 1 -- 172.21.15.119:0/2529848776 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fb02c0036a0 con 0x7fb03c077cc0 2023-01-25T11:40:54.483 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.009+0000 7fb03b7fe700 1 -- 172.21.15.119:0/2529848776 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb02c011870 con 0x7fb03c077cc0 2023-01-25T11:40:54.483 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 -- 172.21.15.119:0/2529848776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 msgr2=0x7fb03c106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:54.483 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 --2- 172.21.15.119:0/2529848776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c106980 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb02c0098a0 tx=0x7fb02c009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:54.483 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 -- 172.21.15.119:0/2529848776 shutdown_connections 2023-01-25T11:40:54.483 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 --2- 172.21.15.119:0/2529848776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c106980 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:54.484 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 -- 172.21.15.119:0/2529848776 >> 172.21.15.119:0/2529848776 conn(0x7fb03c0fe2b0 msgr2=0x7fb03c1006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:54.484 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 -- 172.21.15.119:0/2529848776 shutdown_connections 2023-01-25T11:40:54.484 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 -- 172.21.15.119:0/2529848776 wait complete. 2023-01-25T11:40:54.484 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 Processor -- start 2023-01-25T11:40:54.484 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.010+0000 7fb0427a7700 1 -- start start 2023-01-25T11:40:54.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb0427a7700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c1042e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:54.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb0427a7700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03c104820 con 0x7fb03c077cc0 2023-01-25T11:40:54.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb03bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c1042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:54.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb03bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c1042e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36008/0 (socket says 172.21.15.119:36008) 2023-01-25T11:40:54.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb03bfff700 1 -- 172.21.15.119:0/2587617274 learned_addr learned my addr 172.21.15.119:0/2587617274 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:54.485 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb03bfff700 1 -- 172.21.15.119:0/2587617274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb02c009550 con 0x7fb03c077cc0 2023-01-25T11:40:54.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb03bfff700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c1042e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009fb0 tx=0x7fb02c011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:54.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb02c004110 con 0x7fb03c077cc0 2023-01-25T11:40:54.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fb02c01b440 con 0x7fb03c077cc0 2023-01-25T11:40:54.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb02c023400 con 0x7fb03c077cc0 2023-01-25T11:40:54.486 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03c107f30 con 0x7fb03c077cc0 2023-01-25T11:40:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.011+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03c107da0 con 0x7fb03c077cc0 2023-01-25T11:40:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 43498+0+0 (secure 0 0 0) 0x7fb02c017020 con 0x7fb03c077cc0 2023-01-25T11:40:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb039ffb700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2023-01-25T11:40:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb039ffb700 4 mgrc handle_mgr_map Got map version 4 2023-01-25T11:40:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb039ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:54.487 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb039ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb039ffb700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fb0280366f0 0x7fb028038ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb02c049b60 con 0x7fb03c077cc0 2023-01-25T11:40:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c1080c0 con 0x7fb03c077cc0 2023-01-25T11:40:54.488 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.012+0000 7fb033fff700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fb0280366f0 0x7fb028038ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:54.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.013+0000 7fb033fff700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fb0280366f0 0x7fb028038ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb024006fd0 tx=0x7fb024006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:54.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.017+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fb02c0193d0 con 0x7fb03c077cc0 2023-01-25T11:40:54.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:53.252+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb03c061b00 con 0x7fb03c077cc0 2023-01-25T11:40:54.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.026+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 5) v1 ==== 43509+0+0 (secure 0 0 0) 0x7fb02c00f050 con 0x7fb03c077cc0 2023-01-25T11:40:54.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.026+0000 7fb039ffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-01-25T11:40:54.489 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.026+0000 7fb039ffb700 4 mgrc handle_mgr_map Got map version 5 2023-01-25T11:40:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.026+0000 7fb039ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.027+0000 7fb039ffb700 1 -- 172.21.15.119:0/2587617274 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb02c02ce20 con 0x7fb03c077cc0 2023-01-25T11:40:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.029+0000 7fb0427a7700 10 mgrc shutdown 2023-01-25T11:40:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.029+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fb0280366f0 msgr2=0x7fb028038ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:54.490 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.029+0000 7fb0427a7700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fb0280366f0 0x7fb028038ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb024006fd0 tx=0x7fb024006e40 comp rx=0 tx=0).stop 2023-01-25T11:40:54.491 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 msgr2=0x7fb03c1042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:54.491 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c1042e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb02c009fb0 tx=0x7fb02c011a70 comp rx=0 tx=0).stop 2023-01-25T11:40:54.491 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 shutdown_connections 2023-01-25T11:40:54.491 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7fb0280366f0 0x7fb028038ba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:54.491 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 --2- 172.21.15.119:0/2587617274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb03c077cc0 0x7fb03c1042e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:54.492 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 >> 172.21.15.119:0/2587617274 conn(0x7fb03c0fe2b0 msgr2=0x7fb03c0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:54.492 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 shutdown_connections 2023-01-25T11:40:54.492 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.030+0000 7fb0427a7700 1 -- 172.21.15.119:0/2587617274 wait complete. 2023-01-25T11:40:55.390 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:54 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:54.949+0000 7f40442ca200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T11:40:55.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:55 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/2587617274' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-01-25T11:40:55.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:55 smithi119 ceph-mon[102162]: mgrmap e5: a(active, since 8s) 2023-01-25T11:40:55.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:40:55.622 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-01-25T11:40:55.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2023-01-25T11:40:55.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "a", 2023-01-25T11:40:55.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-01-25T11:40:55.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:40:55.623 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.897+0000 7f6e070cf700 1 Processor -- start 2023-01-25T11:40:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.897+0000 7f6e070cf700 1 -- start start 2023-01-25T11:40:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.898+0000 7f6e070cf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00078310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.898+0000 7f6e070cf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e00078850 con 0x7f6e00079e20 2023-01-25T11:40:55.624 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.898+0000 7f6e04e6b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00078310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.898+0000 7f6e04e6b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00078310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36044/0 (socket says 172.21.15.119:36044) 2023-01-25T11:40:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.898+0000 7f6e04e6b700 1 -- 172.21.15.119:0/1922145725 learned_addr learned my addr 172.21.15.119:0/1922145725 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.898+0000 7f6e04e6b700 1 -- 172.21.15.119:0/1922145725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e00078990 con 0x7f6e00079e20 2023-01-25T11:40:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.898+0000 7f6e04e6b700 1 --2- 172.21.15.119:0/1922145725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00078310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098a0 tx=0x7f6df4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4988b0e4b4096193 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6dff7fe700 1 -- 172.21.15.119:0/1922145725 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6df4004160 con 0x7f6e00079e20 2023-01-25T11:40:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6dff7fe700 1 -- 172.21.15.119:0/1922145725 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f6df40036a0 con 0x7f6e00079e20 2023-01-25T11:40:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6dff7fe700 1 -- 172.21.15.119:0/1922145725 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6df4011870 con 0x7f6e00079e20 2023-01-25T11:40:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6e070cf700 1 -- 172.21.15.119:0/1922145725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 msgr2=0x7f6e00078310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6e070cf700 1 --2- 172.21.15.119:0/1922145725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00078310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098a0 tx=0x7f6df4009bb0 comp rx=0 tx=0).stop 2023-01-25T11:40:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6e070cf700 1 -- 172.21.15.119:0/1922145725 shutdown_connections 2023-01-25T11:40:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6e070cf700 1 --2- 172.21.15.119:0/1922145725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00078310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6e070cf700 1 -- 172.21.15.119:0/1922145725 >> 172.21.15.119:0/1922145725 conn(0x7f6e000ff440 msgr2=0x7f6e00101870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6e070cf700 1 -- 172.21.15.119:0/1922145725 shutdown_connections 2023-01-25T11:40:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.899+0000 7f6e070cf700 1 -- 172.21.15.119:0/1922145725 wait complete. 2023-01-25T11:40:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.900+0000 7f6e070cf700 1 Processor -- start 2023-01-25T11:40:55.627 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.900+0000 7f6e070cf700 1 -- start start 2023-01-25T11:40:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.900+0000 7f6e070cf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.900+0000 7f6e070cf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0019a200 con 0x7f6e00079e20 2023-01-25T11:40:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.900+0000 7f6e04e6b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:40:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.900+0000 7f6e04e6b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36060/0 (socket says 172.21.15.119:36060) 2023-01-25T11:40:55.628 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.900+0000 7f6e04e6b700 1 -- 172.21.15.119:0/2718620298 learned_addr learned my addr 172.21.15.119:0/2718620298 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:40:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.901+0000 7f6e04e6b700 1 -- 172.21.15.119:0/2718620298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df4009550 con 0x7f6e00079e20 2023-01-25T11:40:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.901+0000 7f6e04e6b700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00199cc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009fb0 tx=0x7f6df4011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:40:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.901+0000 7f6dfdffb700 1 -- 172.21.15.119:0/2718620298 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6df4004110 con 0x7f6e00079e20 2023-01-25T11:40:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.901+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0010a220 con 0x7f6e00079e20 2023-01-25T11:40:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.901+0000 7f6dfdffb700 1 -- 172.21.15.119:0/2718620298 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f6df401b440 con 0x7f6e00079e20 2023-01-25T11:40:55.629 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.901+0000 7f6dfdffb700 1 -- 172.21.15.119:0/2718620298 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6df4023400 con 0x7f6e00079e20 2023-01-25T11:40:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.901+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0010a090 con 0x7f6e00079e20 2023-01-25T11:40:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.902+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0010a3b0 con 0x7f6e00079e20 2023-01-25T11:40:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfdffb700 1 -- 172.21.15.119:0/2718620298 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43509+0+0 (secure 0 0 0) 0x7f6df4017020 con 0x7f6e00079e20 2023-01-25T11:40:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfdffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-01-25T11:40:55.630 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfdffb700 4 mgrc handle_mgr_map Got map version 5 2023-01-25T11:40:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfdffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfdffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:40:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfdffb700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 0x7f6df0038bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:40:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfdffb700 1 -- 172.21.15.119:0/2718620298 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6df4049e40 con 0x7f6e00079e20 2023-01-25T11:40:55.631 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfffff700 1 -- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 msgr2=0x7f6df0038bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2579011553 2023-01-25T11:40:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.903+0000 7f6dfffff700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 0x7f6df0038bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T11:40:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:54.907+0000 7f6dfdffb700 1 -- 172.21.15.119:0/2718620298 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f6df402b960 con 0x7f6e00079e20 2023-01-25T11:40:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.104+0000 7f6dfffff700 1 -- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 msgr2=0x7f6df0038bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2579011553 2023-01-25T11:40:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.104+0000 7f6dfffff700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 0x7f6df0038bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T11:40:55.632 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.154+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f6e0019d800 con 0x7f6e00079e20 2023-01-25T11:40:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.154+0000 7f6dfdffb700 1 -- 172.21.15.119:0/2718620298 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f6df402bb70 con 0x7f6e00079e20 2023-01-25T11:40:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 10 mgrc shutdown 2023-01-25T11:40:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 msgr2=0x7f6df0038bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-01-25T11:40:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 0x7f6df0038bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:55.633 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 msgr2=0x7f6e00199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:40:55.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00199cc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009fb0 tx=0x7f6df4011a70 comp rx=0 tx=0).stop 2023-01-25T11:40:55.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 shutdown_connections 2023-01-25T11:40:55.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f6df0036700 0x7f6df0038bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:55.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 --2- 172.21.15.119:0/2718620298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e00079e20 0x7f6e00199cc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:40:55.634 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 >> 172.21.15.119:0/2718620298 conn(0x7f6e000ff440 msgr2=0x7f6e00100220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:40:55.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 shutdown_connections 2023-01-25T11:40:55.635 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:55.156+0000 7f6e070cf700 1 -- 172.21.15.119:0/2718620298 wait complete. 2023-01-25T11:40:55.635 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2023-01-25T11:40:55.635 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 5... 2023-01-25T11:40:55.796 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:55 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:55.519+0000 7f40442ca200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T11:40:55.796 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:55 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:55.609+0000 7f40442ca200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T11:40:56.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:40:56 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/2718620298' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-01-25T11:40:56.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:55 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:55.793+0000 7f40442ca200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T11:40:56.890 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:56 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:56.604+0000 7f40442ca200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T11:40:56.890 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:56 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:56.786+0000 7f40442ca200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T11:40:57.296 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:57 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:57.038+0000 7f40442ca200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T11:40:57.296 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:57 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:57.124+0000 7f40442ca200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T11:40:57.583 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:57 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:57.293+0000 7f40442ca200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T11:40:57.890 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:57 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:57.580+0000 7f40442ca200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T11:40:57.890 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:57 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:57.670+0000 7f40442ca200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T11:40:58.289 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:57 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:57.985+0000 7f40442ca200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T11:40:58.290 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:58 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:58.115+0000 7f40442ca200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T11:40:58.640 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:58 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:58.287+0000 7f40442ca200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T11:40:58.641 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:58 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:58.380+0000 7f40442ca200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T11:40:58.641 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:58 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:58.475+0000 7f40442ca200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T11:40:59.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:58 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:58.749+0000 7f40442ca200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T11:40:59.731 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:59 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:59.444+0000 7f40442ca200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T11:40:59.731 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:59 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:59.541+0000 7f40442ca200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T11:40:59.732 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:59 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:59.631+0000 7f40442ca200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T11:41:00.442 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:40:59 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:40:59.728+0000 7f40442ca200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T11:41:00.442 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:00 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:00.169+0000 7f40442ca200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T11:41:00.442 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:00 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:00.256+0000 7f40442ca200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T11:41:00.890 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:00 smithi119 ceph-mon[102162]: Active manager daemon a restarted 2023-01-25T11:41:00.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:00 smithi119 ceph-mon[102162]: Activating manager daemon a 2023-01-25T11:41:00.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:00 smithi119 ceph-mon[102162]: osdmap e2: 0 total, 0 up, 0 in 2023-01-25T11:41:00.891 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:00 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:00.439+0000 7f40442ca200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T11:41:01.890 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: mgrmap e6: a(active, starting, since 0.0536305s) 2023-01-25T11:41:01.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-01-25T11:41:01.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "mgr metadata", "who": "a", "id": "a"}]: dispatch 2023-01-25T11:41:01.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-01-25T11:41:01.891 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-01-25T11:41:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-01-25T11:41:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: Manager daemon a is now available 2023-01-25T11:41:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: Found migration_current of "None". Setting to last migration. 2023-01-25T11:41:01.892 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:01.893 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/a/mirror_snapshot_schedule"}]: dispatch 2023-01-25T11:41:01.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/a/trash_purge_schedule"}]: dispatch 2023-01-25T11:41:01.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:01.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:01.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:01.894 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:01 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:01.978 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:41:01.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-01-25T11:41:01.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2023-01-25T11:41:01.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:41:01.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0797114700 1 Processor -- start 2023-01-25T11:41:01.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0797114700 1 -- start start 2023-01-25T11:41:01.979 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0797114700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790105980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:01.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0797114700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790105f50 con 0x7f0790105570 2023-01-25T11:41:01.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0794eb0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790105980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:01.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0794eb0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790105980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36066/0 (socket says 172.21.15.119:36066) 2023-01-25T11:41:01.980 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0794eb0700 1 -- 172.21.15.119:0/143866053 learned_addr learned my addr 172.21.15.119:0/143866053 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:01.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.053+0000 7f0794eb0700 1 -- 172.21.15.119:0/143866053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0790101e20 con 0x7f0790105570 2023-01-25T11:41:01.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f0794eb0700 1 --2- 172.21.15.119:0/143866053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790105980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07840098a0 tx=0x7f0784009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd773c48c8549e3b server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:01.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f078f7fe700 1 -- 172.21.15.119:0/143866053 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0784004160 con 0x7f0790105570 2023-01-25T11:41:01.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f078f7fe700 1 -- 172.21.15.119:0/143866053 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f07840036a0 con 0x7f0790105570 2023-01-25T11:41:01.981 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f078f7fe700 1 -- 172.21.15.119:0/143866053 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0784011870 con 0x7f0790105570 2023-01-25T11:41:01.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f0797114700 1 -- 172.21.15.119:0/143866053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 msgr2=0x7f0790105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:01.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f0797114700 1 --2- 172.21.15.119:0/143866053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790105980 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07840098a0 tx=0x7f0784009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f0797114700 1 -- 172.21.15.119:0/143866053 shutdown_connections 2023-01-25T11:41:01.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f0797114700 1 --2- 172.21.15.119:0/143866053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790105980 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f0797114700 1 -- 172.21.15.119:0/143866053 >> 172.21.15.119:0/143866053 conn(0x7f07900fd9b0 msgr2=0x7f07900ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:01.982 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.054+0000 7f0797114700 1 -- 172.21.15.119:0/143866053 shutdown_connections 2023-01-25T11:41:01.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0797114700 1 -- 172.21.15.119:0/143866053 wait complete. 2023-01-25T11:41:01.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0797114700 1 Processor -- start 2023-01-25T11:41:01.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0797114700 1 -- start start 2023-01-25T11:41:01.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0797114700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790104ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:01.983 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0797114700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0790104ff0 con 0x7f0790105570 2023-01-25T11:41:01.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0794eb0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790104ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:01.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0794eb0700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790104ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36078/0 (socket says 172.21.15.119:36078) 2023-01-25T11:41:01.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.055+0000 7f0794eb0700 1 -- 172.21.15.119:0/1613873086 learned_addr learned my addr 172.21.15.119:0/1613873086 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:01.984 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.056+0000 7f0794eb0700 1 -- 172.21.15.119:0/1613873086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784009550 con 0x7f0790105570 2023-01-25T11:41:01.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.056+0000 7f0794eb0700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790104ab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0784000c00 tx=0x7f0784011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:01.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.056+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0784004110 con 0x7f0790105570 2023-01-25T11:41:01.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.056+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f078401b440 con 0x7f0790105570 2023-01-25T11:41:01.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.056+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07840193f0 con 0x7f0790105570 2023-01-25T11:41:01.985 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.056+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0790108370 con 0x7f0790105570 2023-01-25T11:41:01.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.056+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0790102bc0 con 0x7f0790105570 2023-01-25T11:41:01.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f0797114700 20 mgrc start_tell_command target: cmd: [{"prefix": "get_command_descriptions"}] 2023-01-25T11:41:01.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f0797114700 5 mgrc start_tell_command no mgr session (no running mgr daemon?), or not active mgr, waiting 2023-01-25T11:41:01.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 43509+0+0 (secure 0 0 0) 0x7f078402b070 con 0x7f0790105570 2023-01-25T11:41:01.986 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-01-25T11:41:01.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 4 mgrc handle_mgr_map Got map version 5 2023-01-25T11:41:01.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:41:01.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] 2023-01-25T11:41:01.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:01.987 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 10 mgrc reconnect resending 0 (tell) 2023-01-25T11:41:01.988 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 --> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0780039380 con 0x7f07800367c0 2023-01-25T11:41:01.988 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0784049a70 con 0x7f0790105570 2023-01-25T11:41:01.988 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078ffff700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 msgr2=0x7f0780038c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2579011553 2023-01-25T11:41:01.988 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.057+0000 7f078ffff700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T11:41:01.988 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.258+0000 7f078ffff700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 msgr2=0x7f0780038c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2579011553 2023-01-25T11:41:01.989 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.258+0000 7f078ffff700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T11:41:01.989 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.659+0000 7f078ffff700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 msgr2=0x7f0780038c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2579011553 2023-01-25T11:41:01.989 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:56.659+0000 7f078ffff700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-01-25T11:41:01.989 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:57.460+0000 7f078ffff700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 msgr2=0x7f0780038c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2579011553 2023-01-25T11:41:01.989 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:57.460+0000 7f078ffff700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-01-25T11:41:01.990 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:59.061+0000 7f078ffff700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 msgr2=0x7f0780038c70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/2579011553 2023-01-25T11:41:01.990 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:40:59.061+0000 7f078ffff700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-01-25T11:41:01.990 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 6) v1 ==== 43319+0+0 (secure 0 0 0) 0x7f0784021030 con 0x7f0790105570 2023-01-25T11:41:01.990 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 20 mgrc handle_mgr_map mgrmap(e 6) v1 2023-01-25T11:41:01.990 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 4 mgrc handle_mgr_map Got map version 6 2023-01-25T11:41:01.991 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:41:01.991 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 4 mgrc reconnect Terminating session with v2:172.21.15.119:6800/2579011553 2023-01-25T11:41:01.991 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 msgr2=0x7f0780038c70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-01-25T11:41:01.991 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.991 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:00.505+0000 7f078dffb700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:41:01.992 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 7) v1 ==== 43429+0+0 (secure 0 0 0) 0x7f0784017020 con 0x7f0790105570 2023-01-25T11:41:01.992 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 20 mgrc handle_mgr_map mgrmap(e 7) v1 2023-01-25T11:41:01.992 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 4 mgrc handle_mgr_map Got map version 7 2023-01-25T11:41:01.992 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:01.992 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:01.993 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0780039fb0 0x7f078003c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:01.993 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 10 mgrc reconnect resending 0 (tell) 2023-01-25T11:41:01.993 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.510+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0780039380 con 0x7f0780039fb0 2023-01-25T11:41:01.993 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.511+0000 7f078ffff700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0780039fb0 0x7f078003c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:01.993 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.511+0000 7f078ffff700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0780039fb0 0x7f078003c390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f077c003a10 tx=0x7f077c0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:01.994 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.512+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f0780039380 con 0x7f0780039fb0 2023-01-25T11:41:01.994 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.512+0000 7f078dffb700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:01.994 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.518+0000 7f0797114700 20 mgrc start_tell_command target: cmd: [{"prefix": "mgr_status"}] 2023-01-25T11:41:01.994 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.518+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0790104640 con 0x7f0780039fb0 2023-01-25T11:41:01.994 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.518+0000 7f078dffb700 1 -- 172.21.15.119:0/1613873086 <== mgr.14120 v2:172.21.15.119:6800/569691540 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0790104640 con 0x7f0780039fb0 2023-01-25T11:41:01.995 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.518+0000 7f078dffb700 20 mgrc handle_command_reply tid 1 r 0 2023-01-25T11:41:01.995 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 10 mgrc shutdown 2023-01-25T11:41:01.995 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0780039fb0 msgr2=0x7f078003c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:01.995 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0780039fb0 0x7f078003c390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f077c003a10 tx=0x7f077c0092b0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.995 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 msgr2=0x7f0790104ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:01.996 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790104ab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0784000c00 tx=0x7f0784011ae0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.996 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 shutdown_connections 2023-01-25T11:41:01.996 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0780039fb0 0x7f078003c390 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.996 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0790105570 0x7f0790104ab0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.996 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 --2- 172.21.15.119:0/1613873086 >> [v2:172.21.15.119:6800/2579011553,v1:172.21.15.119:6801/2579011553] conn(0x7f07800367c0 0x7f0780038c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:01.997 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 >> 172.21.15.119:0/1613873086 conn(0x7f07900fd9b0 msgr2=0x7f07900fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:01.997 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 shutdown_connections 2023-01-25T11:41:01.997 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:01.519+0000 7f0797114700 1 -- 172.21.15.119:0/1613873086 wait complete. 2023-01-25T11:41:01.997 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 5 is available 2023-01-25T11:41:01.997 INFO:teuthology.orchestra.run.smithi119.stdout:Setting orchestrator backend to cephadm... 2023-01-25T11:41:02.827 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-01-25T11:41:02.828 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: mgrmap e7: a(active, since 1.05916s) 2023-01-25T11:41:02.828 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-01-25T11:41:02.828 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:01] ENGINE Bus STARTING 2023-01-25T11:41:02.829 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:01] ENGINE Serving on https://172.21.15.119:8765 2023-01-25T11:41:02.829 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:01] ENGINE Serving on https://172.21.15.119:7150 2023-01-25T11:41:02.829 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:01] ENGINE Bus STARTED 2023-01-25T11:41:02.829 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:02 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:03.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.389+0000 7f7b2bda6700 1 Processor -- start 2023-01-25T11:41:03.115 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.389+0000 7f7b2bda6700 1 -- start start 2023-01-25T11:41:03.116 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.389+0000 7f7b2bda6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:03.116 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.389+0000 7f7b2bda6700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b24107160 con 0x7f7b24104840 2023-01-25T11:41:03.116 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.389+0000 7f7b29b42700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:03.116 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.389+0000 7f7b29b42700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59740/0 (socket says 172.21.15.119:59740) 2023-01-25T11:41:03.116 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.389+0000 7f7b29b42700 1 -- 172.21.15.119:0/1679367891 learned_addr learned my addr 172.21.15.119:0/1679367891 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:03.117 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b29b42700 1 -- 172.21.15.119:0/1679367891 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b241072a0 con 0x7f7b24104840 2023-01-25T11:41:03.117 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b29b42700 1 --2- 172.21.15.119:0/1679367891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24106c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b200098a0 tx=0x7f7b20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e0fa1efb077c57 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:03.117 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b28b40700 1 -- 172.21.15.119:0/1679367891 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b20004160 con 0x7f7b24104840 2023-01-25T11:41:03.117 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b28b40700 1 -- 172.21.15.119:0/1679367891 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f7b200036a0 con 0x7f7b24104840 2023-01-25T11:41:03.117 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b28b40700 1 -- 172.21.15.119:0/1679367891 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b20011870 con 0x7f7b24104840 2023-01-25T11:41:03.118 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1679367891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 msgr2=0x7f7b24106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:03.118 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b2bda6700 1 --2- 172.21.15.119:0/1679367891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24106c20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b200098a0 tx=0x7f7b20009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:03.118 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1679367891 shutdown_connections 2023-01-25T11:41:03.118 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b2bda6700 1 --2- 172.21.15.119:0/1679367891 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24106c20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:03.118 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.390+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1679367891 >> 172.21.15.119:0/1679367891 conn(0x7f7b240fe220 msgr2=0x7f7b24100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:03.119 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.391+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1679367891 shutdown_connections 2023-01-25T11:41:03.119 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.391+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1679367891 wait complete. 2023-01-25T11:41:03.119 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.391+0000 7f7b2bda6700 1 Processor -- start 2023-01-25T11:41:03.119 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.391+0000 7f7b2bda6700 1 -- start start 2023-01-25T11:41:03.119 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b2bda6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:03.119 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b2bda6700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b24198060 con 0x7f7b24104840 2023-01-25T11:41:03.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b29b42700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:03.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b29b42700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59750/0 (socket says 172.21.15.119:59750) 2023-01-25T11:41:03.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b29b42700 1 -- 172.21.15.119:0/1482123423 learned_addr learned my addr 172.21.15.119:0/1482123423 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:03.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b29b42700 1 -- 172.21.15.119:0/1482123423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20009550 con 0x7f7b24104840 2023-01-25T11:41:03.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b29b42700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24197b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b200119c0 tx=0x7f7b20011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b1affd700 1 -- 172.21.15.119:0/1482123423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b20004110 con 0x7f7b24104840 2023-01-25T11:41:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b1affd700 1 -- 172.21.15.119:0/1482123423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f7b2001b440 con 0x7f7b24104840 2023-01-25T11:41:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b24108110 con 0x7f7b24104840 2023-01-25T11:41:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b1affd700 1 -- 172.21.15.119:0/1482123423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b20023400 con 0x7f7b24104840 2023-01-25T11:41:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.392+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b24107f80 con 0x7f7b24104840 2023-01-25T11:41:03.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b1affd700 1 -- 172.21.15.119:0/1482123423 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 7) v1 ==== 43429+0+0 (secure 0 0 0) 0x7f7b20017020 con 0x7f7b24104840 2023-01-25T11:41:03.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b1affd700 20 mgrc handle_mgr_map mgrmap(e 7) v1 2023-01-25T11:41:03.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b1affd700 4 mgrc handle_mgr_map Got map version 7 2023-01-25T11:41:03.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b1affd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:03.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b1affd700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:03.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b1affd700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f7b100366c0 0x7f7b10038b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:03.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b1affd700 1 -- 172.21.15.119:0/1482123423 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7b20049b80 con 0x7f7b24104840 2023-01-25T11:41:03.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b29341700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f7b100366c0 0x7f7b10038b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:03.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.393+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b241082a0 con 0x7f7b24104840 2023-01-25T11:41:03.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.394+0000 7f7b29341700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f7b100366c0 0x7f7b10038b70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b14006fd0 tx=0x7f7b14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:03.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.398+0000 7f7b1affd700 1 -- 172.21.15.119:0/1482123423 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f7b20049e60 con 0x7f7b24104840 2023-01-25T11:41:03.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.604+0000 7f7b2bda6700 20 mgrc start_command cmd: [{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}] 2023-01-25T11:41:03.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.604+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b24198e50 con 0x7f7b100366c0 2023-01-25T11:41:03.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.609+0000 7f7b1affd700 1 -- 172.21.15.119:0/1482123423 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7b24198e50 con 0x7f7b100366c0 2023-01-25T11:41:03.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.609+0000 7f7b1affd700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:03.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 10 mgrc shutdown 2023-01-25T11:41:03.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f7b100366c0 msgr2=0x7f7b10038b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:03.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f7b100366c0 0x7f7b10038b70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b14006fd0 tx=0x7f7b14006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:03.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 msgr2=0x7f7b24197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:03.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24197b20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7b200119c0 tx=0x7f7b20011b30 comp rx=0 tx=0).stop 2023-01-25T11:41:03.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 shutdown_connections 2023-01-25T11:41:03.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f7b100366c0 0x7f7b10038b70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:03.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 --2- 172.21.15.119:0/1482123423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b24104840 0x7f7b24197b20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:03.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.611+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 >> 172.21.15.119:0/1482123423 conn(0x7f7b240fe220 msgr2=0x7f7b240feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:03.127 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.612+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 shutdown_connections 2023-01-25T11:41:03.127 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:02.612+0000 7f7b2bda6700 1 -- 172.21.15.119:0/1482123423 wait complete. 2023-01-25T11:41:03.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:03 smithi119 ceph-mon[102162]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:03.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:03 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:03.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:03 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:03.877 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:03 smithi119 ceph-mon[102162]: mgrmap e8: a(active, since 2s) 2023-01-25T11:41:04.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout value unchanged 2023-01-25T11:41:04.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.506+0000 7fecc0aae700 1 Processor -- start 2023-01-25T11:41:04.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.506+0000 7fecc0aae700 1 -- start start 2023-01-25T11:41:04.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.506+0000 7fecc0aae700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc105910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:04.155 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.506+0000 7fecc0aae700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc105ee0 con 0x7fecbc105500 2023-01-25T11:41:04.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.506+0000 7fecba59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc105910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:04.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.506+0000 7fecba59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc105910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59754/0 (socket says 172.21.15.119:59754) 2023-01-25T11:41:04.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.506+0000 7fecba59c700 1 -- 172.21.15.119:0/147589732 learned_addr learned my addr 172.21.15.119:0/147589732 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:04.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecba59c700 1 -- 172.21.15.119:0/147589732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc101d40 con 0x7fecbc105500 2023-01-25T11:41:04.156 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecba59c700 1 --2- 172.21.15.119:0/147589732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc105910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fecac0098a0 tx=0x7fecac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6cffd29e6d6867 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:04.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecb9d9b700 1 -- 172.21.15.119:0/147589732 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac004160 con 0x7fecbc105500 2023-01-25T11:41:04.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecb9d9b700 1 -- 172.21.15.119:0/147589732 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fecac0036a0 con 0x7fecbc105500 2023-01-25T11:41:04.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecb9d9b700 1 -- 172.21.15.119:0/147589732 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac011870 con 0x7fecbc105500 2023-01-25T11:41:04.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecc0aae700 1 -- 172.21.15.119:0/147589732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 msgr2=0x7fecbc105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:04.157 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecc0aae700 1 --2- 172.21.15.119:0/147589732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc105910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fecac0098a0 tx=0x7fecac009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:04.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecc0aae700 1 -- 172.21.15.119:0/147589732 shutdown_connections 2023-01-25T11:41:04.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecc0aae700 1 --2- 172.21.15.119:0/147589732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc105910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:04.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.507+0000 7fecc0aae700 1 -- 172.21.15.119:0/147589732 >> 172.21.15.119:0/147589732 conn(0x7fecbc0fd9b0 msgr2=0x7fecbc0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:04.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.508+0000 7fecc0aae700 1 -- 172.21.15.119:0/147589732 shutdown_connections 2023-01-25T11:41:04.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.508+0000 7fecc0aae700 1 -- 172.21.15.119:0/147589732 wait complete. 2023-01-25T11:41:04.158 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.508+0000 7fecc0aae700 1 Processor -- start 2023-01-25T11:41:04.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.508+0000 7fecc0aae700 1 -- start start 2023-01-25T11:41:04.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecc0aae700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:04.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecc0aae700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc198080 con 0x7fecbc105500 2023-01-25T11:41:04.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecba59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:04.159 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecba59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc197b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59768/0 (socket says 172.21.15.119:59768) 2023-01-25T11:41:04.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecba59c700 1 -- 172.21.15.119:0/954525106 learned_addr learned my addr 172.21.15.119:0/954525106 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:04.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecba59c700 1 -- 172.21.15.119:0/954525106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac009550 con 0x7fecbc105500 2023-01-25T11:41:04.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecba59c700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc197b40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fecac003b60 tx=0x7fecac011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:04.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecb3fff700 1 -- 172.21.15.119:0/954525106 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac004110 con 0x7fecbc105500 2023-01-25T11:41:04.160 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecb3fff700 1 -- 172.21.15.119:0/954525106 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fecac01b440 con 0x7fecbc105500 2023-01-25T11:41:04.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc108290 con 0x7fecbc105500 2023-01-25T11:41:04.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecb3fff700 1 -- 172.21.15.119:0/954525106 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fecac0193d0 con 0x7fecbc105500 2023-01-25T11:41:04.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.509+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc102af0 con 0x7fecbc105500 2023-01-25T11:41:04.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb3fff700 1 -- 172.21.15.119:0/954525106 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7fecac02a070 con 0x7fecbc105500 2023-01-25T11:41:04.161 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb3fff700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:04.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb3fff700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:04.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb3fff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:04.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb3fff700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:04.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb3fff700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7feca8036750 0x7feca8038c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:04.162 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb3fff700 1 -- 172.21.15.119:0/954525106 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fecac049d30 con 0x7fecbc105500 2023-01-25T11:41:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc108420 con 0x7fecbc105500 2023-01-25T11:41:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.510+0000 7fecb1d9b700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7feca8036750 0x7feca8038c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.511+0000 7fecb1d9b700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7feca8036750 0x7feca8038c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feca4006fd0 tx=0x7feca4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.515+0000 7fecb3fff700 1 -- 172.21.15.119:0/954525106 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fecac018940 con 0x7fecbc105500 2023-01-25T11:41:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.706+0000 7fecc0aae700 20 mgrc start_command cmd: [{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}] 2023-01-25T11:41:04.163 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.706+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fecbc061b00 con 0x7feca8036750 2023-01-25T11:41:04.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.706+0000 7fecb3fff700 1 -- 172.21.15.119:0/954525106 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fecbc061b00 con 0x7feca8036750 2023-01-25T11:41:04.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.706+0000 7fecb3fff700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:04.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 10 mgrc shutdown 2023-01-25T11:41:04.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7feca8036750 msgr2=0x7feca8038c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:04.164 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7feca8036750 0x7feca8038c00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feca4006fd0 tx=0x7feca4006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:04.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 msgr2=0x7fecbc197b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:04.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc197b40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fecac003b60 tx=0x7fecac011a50 comp rx=0 tx=0).stop 2023-01-25T11:41:04.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 shutdown_connections 2023-01-25T11:41:04.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7feca8036750 0x7feca8038c00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:04.165 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 --2- 172.21.15.119:0/954525106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecbc105500 0x7fecbc197b40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:04.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 >> 172.21.15.119:0/954525106 conn(0x7fecbc0fd9b0 msgr2=0x7fecbc0fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:04.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 shutdown_connections 2023-01-25T11:41:04.166 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:03.708+0000 7fecc0aae700 1 -- 172.21.15.119:0/954525106 wait complete. 2023-01-25T11:41:04.166 INFO:teuthology.orchestra.run.smithi119.stdout:Generating ssh key... 2023-01-25T11:41:04.821 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-mon[102162]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:05.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: Generating public/private rsa key pair. 2023-01-25T11:41:05.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: Your identification has been saved in /tmp/tmpdiozilm2/key. 2023-01-25T11:41:05.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: Your public key has been saved in /tmp/tmpdiozilm2/key.pub. 2023-01-25T11:41:05.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: The key fingerprint is: 2023-01-25T11:41:05.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: SHA256:AvS8if3Z3SqNUFM6kni4cCoGfiF5xCO7OeRHlyo/+Wo ceph-f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:41:05.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: The key's randomart image is: 2023-01-25T11:41:05.142 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: +---[RSA 3072]----+ 2023-01-25T11:41:05.142 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: | . . | 2023-01-25T11:41:05.142 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: | . = o . | 2023-01-25T11:41:05.142 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: | = o +o . o | 2023-01-25T11:41:05.142 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: |.= +.*+o+ = | 2023-01-25T11:41:05.143 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: |+.* =+=oSo o | 2023-01-25T11:41:05.143 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: | Bo+. .o.o . . | 2023-01-25T11:41:05.143 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: | .*.. o..o. . | 2023-01-25T11:41:05.143 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: | E o .. | 2023-01-25T11:41:05.143 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: | ..+. .. | 2023-01-25T11:41:05.144 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:04 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: +----[SHA256]-----+ 2023-01-25T11:41:05.293 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6da38c700 1 Processor -- start 2023-01-25T11:41:05.293 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6da38c700 1 -- start start 2023-01-25T11:41:05.293 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6da38c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d4106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:05.294 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6da38c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d4077ed0 con 0x7fb6d4104840 2023-01-25T11:41:05.294 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d4106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:05.294 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d4106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59780/0 (socket says 172.21.15.119:59780) 2023-01-25T11:41:05.294 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6d3fff700 1 -- 172.21.15.119:0/1602779864 learned_addr learned my addr 172.21.15.119:0/1602779864 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:05.294 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6d3fff700 1 -- 172.21.15.119:0/1602779864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d4107160 con 0x7fb6d4104840 2023-01-25T11:41:05.295 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.581+0000 7fb6d3fff700 1 --2- 172.21.15.119:0/1602779864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d4106c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40098a0 tx=0x7fb6c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73666ab544377795 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:05.295 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6d2ffd700 1 -- 172.21.15.119:0/1602779864 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c4004160 con 0x7fb6d4104840 2023-01-25T11:41:05.295 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6d2ffd700 1 -- 172.21.15.119:0/1602779864 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fb6c40036a0 con 0x7fb6d4104840 2023-01-25T11:41:05.295 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6d2ffd700 1 -- 172.21.15.119:0/1602779864 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c4011870 con 0x7fb6d4104840 2023-01-25T11:41:05.295 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6da38c700 1 -- 172.21.15.119:0/1602779864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 msgr2=0x7fb6d4106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:05.296 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6da38c700 1 --2- 172.21.15.119:0/1602779864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d4106c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb6c40098a0 tx=0x7fb6c4009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:05.296 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6da38c700 1 -- 172.21.15.119:0/1602779864 shutdown_connections 2023-01-25T11:41:05.296 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6da38c700 1 --2- 172.21.15.119:0/1602779864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d4106c20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:05.296 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6da38c700 1 -- 172.21.15.119:0/1602779864 >> 172.21.15.119:0/1602779864 conn(0x7fb6d40fe220 msgr2=0x7fb6d4100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:05.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6da38c700 1 -- 172.21.15.119:0/1602779864 shutdown_connections 2023-01-25T11:41:05.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.582+0000 7fb6da38c700 1 -- 172.21.15.119:0/1602779864 wait complete. 2023-01-25T11:41:05.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.583+0000 7fb6da38c700 1 Processor -- start 2023-01-25T11:41:05.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.583+0000 7fb6da38c700 1 -- start start 2023-01-25T11:41:05.297 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.583+0000 7fb6da38c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d41041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:05.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.583+0000 7fb6da38c700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d41046f0 con 0x7fb6d4104840 2023-01-25T11:41:05.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.583+0000 7fb6d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d41041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:05.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.583+0000 7fb6d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d41041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59794/0 (socket says 172.21.15.119:59794) 2023-01-25T11:41:05.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.583+0000 7fb6d3fff700 1 -- 172.21.15.119:0/2700670761 learned_addr learned my addr 172.21.15.119:0/2700670761 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:05.298 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.584+0000 7fb6d3fff700 1 -- 172.21.15.119:0/2700670761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c4009550 con 0x7fb6d4104840 2023-01-25T11:41:05.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.584+0000 7fb6d3fff700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d41041b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4000c00 tx=0x7fb6c4011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:05.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.584+0000 7fb6d17fa700 1 -- 172.21.15.119:0/2700670761 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c4004110 con 0x7fb6d4104840 2023-01-25T11:41:05.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.584+0000 7fb6d17fa700 1 -- 172.21.15.119:0/2700670761 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fb6c401b440 con 0x7fb6d4104840 2023-01-25T11:41:05.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.584+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d4108030 con 0x7fb6d4104840 2023-01-25T11:41:05.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.584+0000 7fb6d17fa700 1 -- 172.21.15.119:0/2700670761 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6c40193f0 con 0x7fb6d4104840 2023-01-25T11:41:05.299 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.584+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d4107ea0 con 0x7fb6d4104840 2023-01-25T11:41:05.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d17fa700 1 -- 172.21.15.119:0/2700670761 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7fb6c402b070 con 0x7fb6d4104840 2023-01-25T11:41:05.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d17fa700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:05.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d17fa700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:05.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d17fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:05.300 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d17fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:05.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d17fa700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fb6bc0367c0 0x7fb6bc038c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:05.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d17fa700 1 -- 172.21.15.119:0/2700670761 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb6c4049af0 con 0x7fb6d4104840 2023-01-25T11:41:05.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d37fe700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fb6bc0367c0 0x7fb6bc038c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:05.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d41081c0 con 0x7fb6d4104840 2023-01-25T11:41:05.301 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.585+0000 7fb6d37fe700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fb6bc0367c0 0x7fb6bc038c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8006fd0 tx=0x7fb6c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:05.302 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.590+0000 7fb6d17fa700 1 -- 172.21.15.119:0/2700670761 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fb6c401f070 con 0x7fb6d4104840 2023-01-25T11:41:05.302 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.780+0000 7fb6da38c700 20 mgrc start_command cmd: [{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}] 2023-01-25T11:41:05.302 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.780+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6d4061b00 con 0x7fb6bc0367c0 2023-01-25T11:41:05.302 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.826+0000 7fb6d17fa700 1 -- 172.21.15.119:0/2700670761 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb6d4061b00 con 0x7fb6bc0367c0 2023-01-25T11:41:05.302 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.826+0000 7fb6d17fa700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:05.303 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 10 mgrc shutdown 2023-01-25T11:41:05.303 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fb6bc0367c0 msgr2=0x7fb6bc038c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:05.303 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fb6bc0367c0 0x7fb6bc038c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8006fd0 tx=0x7fb6c8006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:05.303 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 msgr2=0x7fb6d41041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:05.304 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d41041b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4000c00 tx=0x7fb6c4011ae0 comp rx=0 tx=0).stop 2023-01-25T11:41:05.304 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 shutdown_connections 2023-01-25T11:41:05.304 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fb6bc0367c0 0x7fb6bc038c70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:05.304 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 --2- 172.21.15.119:0/2700670761 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d4104840 0x7fb6d41041b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:05.304 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 >> 172.21.15.119:0/2700670761 conn(0x7fb6d40fe220 msgr2=0x7fb6d40feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:05.305 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 shutdown_connections 2023-01-25T11:41:05.305 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:04.828+0000 7fb6da38c700 1 -- 172.21.15.119:0/2700670761 wait complete. 2023-01-25T11:41:06.093 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:05 smithi119 ceph-mon[102162]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:06.094 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:05 smithi119 ceph-mon[102162]: Generating ssh key... 2023-01-25T11:41:06.094 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:05 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:06.094 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:05 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:06.332 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMmcyM7xJbTW+5yUJFO/Qy6TKrBcAEmqlgGFf4bMs0ViSVeAe3cOWr4iZxJLAcl2gAaoWCZSzzXm28kd8G5mrgn4YFP5EMjvlEGFwYIOE+30Nim52GGvByyryPHjGOmFLiSS56pCRhU3gGHa1GbPXrDjqk6i5V47CAKqgleNrG4FKF1C8lG+5DjVdL+zkrV6tGQS5qJaVWa5rzb6yN+3IUADfBPbT828Z8EiQUPFK6q0WDKyvLWuyJio+PgXogJa8XbFoEc7uIoGj22eAtEKWRYM9KnLRcBfVg/HyDg8e6jDcScD/7ZP0R98P/ICM9AAMEfatufN7FFleDOqf3ZbwwrQtou8lFfu7GKaCa1D6MArg4qmrnirFmZJ3Rw2KwzdiutcjLnKVs/GPYUVuj0Bk8noL6wZ2ft3JYzi9x4IFoiMZ0nB82NkVsZ5RdhMBF2/Une4UlkNOFNk76rGa8rOE5+oqNCMPVwTzPvdP2m6p2HcTeuGArFpNrbLLCzonPpfU= ceph-f64523de-9ca4-11ed-9e56-001a4aab830c 2023-01-25T11:41:06.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.707+0000 7f0ca0695700 1 Processor -- start 2023-01-25T11:41:06.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.707+0000 7f0ca0695700 1 -- start start 2023-01-25T11:41:06.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.707+0000 7f0ca0695700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98106a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:06.333 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.707+0000 7f0ca0695700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c98106f50 con 0x7f0c98077d20 2023-01-25T11:41:06.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.707+0000 7f0c9e431700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98106a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:06.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.707+0000 7f0c9e431700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98106a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59802/0 (socket says 172.21.15.119:59802) 2023-01-25T11:41:06.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.707+0000 7f0c9e431700 1 -- 172.21.15.119:0/4047423118 learned_addr learned my addr 172.21.15.119:0/4047423118 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:06.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0c9e431700 1 -- 172.21.15.119:0/4047423118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c98107090 con 0x7f0c98077d20 2023-01-25T11:41:06.334 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0c9e431700 1 --2- 172.21.15.119:0/4047423118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98106a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c940098a0 tx=0x7f0c94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9444b4a8047ae43 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0c9d42f700 1 -- 172.21.15.119:0/4047423118 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c94004160 con 0x7f0c98077d20 2023-01-25T11:41:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0c9d42f700 1 -- 172.21.15.119:0/4047423118 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f0c940036a0 con 0x7f0c98077d20 2023-01-25T11:41:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0c9d42f700 1 -- 172.21.15.119:0/4047423118 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c94011870 con 0x7f0c98077d20 2023-01-25T11:41:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0ca0695700 1 -- 172.21.15.119:0/4047423118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 msgr2=0x7f0c98106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:06.335 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0ca0695700 1 --2- 172.21.15.119:0/4047423118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98106a10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0c940098a0 tx=0x7f0c94009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:06.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0ca0695700 1 -- 172.21.15.119:0/4047423118 shutdown_connections 2023-01-25T11:41:06.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0ca0695700 1 --2- 172.21.15.119:0/4047423118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98106a10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:06.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.708+0000 7f0ca0695700 1 -- 172.21.15.119:0/4047423118 >> 172.21.15.119:0/4047423118 conn(0x7f0c980fe380 msgr2=0x7f0c981007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:06.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0ca0695700 1 -- 172.21.15.119:0/4047423118 shutdown_connections 2023-01-25T11:41:06.336 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0ca0695700 1 -- 172.21.15.119:0/4047423118 wait complete. 2023-01-25T11:41:06.337 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0ca0695700 1 Processor -- start 2023-01-25T11:41:06.337 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0ca0695700 1 -- start start 2023-01-25T11:41:06.337 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0ca0695700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98104370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:06.337 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0ca0695700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c981048b0 con 0x7f0c98077d20 2023-01-25T11:41:06.337 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0c9e431700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98104370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:06.338 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0c9e431700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98104370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59808/0 (socket says 172.21.15.119:59808) 2023-01-25T11:41:06.338 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.709+0000 7f0c9e431700 1 -- 172.21.15.119:0/2391426409 learned_addr learned my addr 172.21.15.119:0/2391426409 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:06.338 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.710+0000 7f0c9e431700 1 -- 172.21.15.119:0/2391426409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c94009550 con 0x7f0c98077d20 2023-01-25T11:41:06.338 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.710+0000 7f0c9e431700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98104370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000c00 tx=0x7f0c94011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:06.338 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.710+0000 7f0c8f7fe700 1 -- 172.21.15.119:0/2391426409 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c94004110 con 0x7f0c98077d20 2023-01-25T11:41:06.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.710+0000 7f0c8f7fe700 1 -- 172.21.15.119:0/2391426409 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f0c9401b440 con 0x7f0c98077d20 2023-01-25T11:41:06.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.710+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c98108020 con 0x7f0c98077d20 2023-01-25T11:41:06.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.710+0000 7f0c8f7fe700 1 -- 172.21.15.119:0/2391426409 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c940193f0 con 0x7f0c98077d20 2023-01-25T11:41:06.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.710+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c98107e90 con 0x7f0c98077d20 2023-01-25T11:41:06.339 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c8f7fe700 1 -- 172.21.15.119:0/2391426409 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7f0c9402b070 con 0x7f0c98077d20 2023-01-25T11:41:06.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c8f7fe700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:06.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c981081b0 con 0x7f0c98077d20 2023-01-25T11:41:06.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c8f7fe700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:06.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c8f7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:06.340 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c8f7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:06.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c8f7fe700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0c840367c0 0x7f0c84038c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:06.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c8f7fe700 1 -- 172.21.15.119:0/2391426409 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0c94049c40 con 0x7f0c98077d20 2023-01-25T11:41:06.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.711+0000 7f0c9dc30700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0c840367c0 0x7f0c84038c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:06.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.712+0000 7f0c9dc30700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0c840367c0 0x7f0c84038c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c88006fd0 tx=0x7f0c88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:06.341 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.716+0000 7f0c8f7fe700 1 -- 172.21.15.119:0/2391426409 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f0c9402c9a0 con 0x7f0c98077d20 2023-01-25T11:41:06.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.907+0000 7f0ca0695700 20 mgrc start_command cmd: [{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}] 2023-01-25T11:41:06.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.907+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c981031f0 con 0x7f0c840367c0 2023-01-25T11:41:06.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.907+0000 7f0c8f7fe700 1 -- 172.21.15.119:0/2391426409 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f0c981031f0 con 0x7f0c840367c0 2023-01-25T11:41:06.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.907+0000 7f0c8f7fe700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:06.342 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 10 mgrc shutdown 2023-01-25T11:41:06.343 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0c840367c0 msgr2=0x7f0c84038c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:06.343 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0c840367c0 0x7f0c84038c70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0c88006fd0 tx=0x7f0c88006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:06.343 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 msgr2=0x7f0c98104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:06.343 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98104370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0c94000c00 tx=0x7f0c94011ae0 comp rx=0 tx=0).stop 2023-01-25T11:41:06.343 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 shutdown_connections 2023-01-25T11:41:06.344 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f0c840367c0 0x7f0c84038c70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:06.344 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 --2- 172.21.15.119:0/2391426409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c98077d20 0x7f0c98104370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:06.344 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 >> 172.21.15.119:0/2391426409 conn(0x7f0c980fe380 msgr2=0x7f0c980ff110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:06.344 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 shutdown_connections 2023-01-25T11:41:06.344 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:05.909+0000 7f0ca0695700 1 -- 172.21.15.119:0/2391426409 wait complete. 2023-01-25T11:41:06.345 INFO:teuthology.orchestra.run.smithi119.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-01-25T11:41:06.345 INFO:teuthology.orchestra.run.smithi119.stdout:Adding key to root@localhost authorized_keys... 2023-01-25T11:41:06.345 INFO:teuthology.orchestra.run.smithi119.stdout:Adding host smithi119... 2023-01-25T11:41:07.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:06 smithi119 ceph-mon[102162]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:08.279 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:08 smithi119 ceph-mon[102162]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Added host 'smithi119' with addr '172.21.15.119' 2023-01-25T11:41:09.176 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a26750700 1 Processor -- start 2023-01-25T11:41:09.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a26750700 1 -- start start 2023-01-25T11:41:09.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a26750700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:09.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a26750700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20077f70 con 0x7f1a201048a0 2023-01-25T11:41:09.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:09.177 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54270/0 (socket says 172.21.15.119:54270) 2023-01-25T11:41:09.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a1ffff700 1 -- 172.21.15.119:0/2249609209 learned_addr learned my addr 172.21.15.119:0/2249609209 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:09.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.730+0000 7f1a1ffff700 1 -- 172.21.15.119:0/2249609209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a201071c0 con 0x7f1a201048a0 2023-01-25T11:41:09.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a1ffff700 1 --2- 172.21.15.119:0/2249609209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20106c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098a0 tx=0x7f1a10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2e327e6737393 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:09.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a1effd700 1 -- 172.21.15.119:0/2249609209 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a10004160 con 0x7f1a201048a0 2023-01-25T11:41:09.178 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a1effd700 1 -- 172.21.15.119:0/2249609209 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f1a100036a0 con 0x7f1a201048a0 2023-01-25T11:41:09.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a1effd700 1 -- 172.21.15.119:0/2249609209 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a10011870 con 0x7f1a201048a0 2023-01-25T11:41:09.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a26750700 1 -- 172.21.15.119:0/2249609209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 msgr2=0x7f1a20106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:09.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a26750700 1 --2- 172.21.15.119:0/2249609209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20106c80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1a100098a0 tx=0x7f1a10009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:09.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a26750700 1 -- 172.21.15.119:0/2249609209 shutdown_connections 2023-01-25T11:41:09.179 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a26750700 1 --2- 172.21.15.119:0/2249609209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20106c80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:09.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a26750700 1 -- 172.21.15.119:0/2249609209 >> 172.21.15.119:0/2249609209 conn(0x7f1a200fe220 msgr2=0x7f1a20100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:09.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a26750700 1 -- 172.21.15.119:0/2249609209 shutdown_connections 2023-01-25T11:41:09.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.731+0000 7f1a26750700 1 -- 172.21.15.119:0/2249609209 wait complete. 2023-01-25T11:41:09.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.732+0000 7f1a26750700 1 Processor -- start 2023-01-25T11:41:09.180 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.732+0000 7f1a26750700 1 -- start start 2023-01-25T11:41:09.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.732+0000 7f1a26750700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20104210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:09.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.732+0000 7f1a26750700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a20104750 con 0x7f1a201048a0 2023-01-25T11:41:09.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.732+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:09.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.732+0000 7f1a1ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20104210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54274/0 (socket says 172.21.15.119:54274) 2023-01-25T11:41:09.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.732+0000 7f1a1ffff700 1 -- 172.21.15.119:0/4059005205 learned_addr learned my addr 172.21.15.119:0/4059005205 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:09.181 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.733+0000 7f1a1ffff700 1 -- 172.21.15.119:0/4059005205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10009550 con 0x7f1a201048a0 2023-01-25T11:41:09.182 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.733+0000 7f1a1ffff700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20104210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1a10003f20 tx=0x7f1a10011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:09.182 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.733+0000 7f1a1d7fa700 1 -- 172.21.15.119:0/4059005205 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a10004110 con 0x7f1a201048a0 2023-01-25T11:41:09.182 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.733+0000 7f1a1d7fa700 1 -- 172.21.15.119:0/4059005205 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f1a1001b440 con 0x7f1a201048a0 2023-01-25T11:41:09.182 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a20108150 con 0x7f1a201048a0 2023-01-25T11:41:09.182 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.733+0000 7f1a1d7fa700 1 -- 172.21.15.119:0/4059005205 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1a100193d0 con 0x7f1a201048a0 2023-01-25T11:41:09.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a20107fc0 con 0x7f1a201048a0 2023-01-25T11:41:09.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a201082e0 con 0x7f1a201048a0 2023-01-25T11:41:09.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1d7fa700 1 -- 172.21.15.119:0/4059005205 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7f1a1002a070 con 0x7f1a201048a0 2023-01-25T11:41:09.183 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1d7fa700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:09.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1d7fa700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:09.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1d7fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:09.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1d7fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:09.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1d7fa700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f1a080367a0 0x7f1a08038c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:09.184 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1d7fa700 1 -- 172.21.15.119:0/4059005205 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1a100499f0 con 0x7f1a201048a0 2023-01-25T11:41:09.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.734+0000 7f1a1f7fe700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f1a080367a0 0x7f1a08038c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:09.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.735+0000 7f1a1f7fe700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f1a080367a0 0x7f1a08038c50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006fd0 tx=0x7f1a14006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:09.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.739+0000 7f1a1d7fa700 1 -- 172.21.15.119:0/4059005205 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f1a1002b760 con 0x7f1a201048a0 2023-01-25T11:41:09.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.933+0000 7f1a26750700 20 mgrc start_command cmd: [{"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}] 2023-01-25T11:41:09.185 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:06.933+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi119", "addr": "172.21.15.119", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a20061b00 con 0x7f1a080367a0 2023-01-25T11:41:09.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.730+0000 7f1a1d7fa700 1 -- 172.21.15.119:0/4059005205 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1a20061b00 con 0x7f1a080367a0 2023-01-25T11:41:09.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.730+0000 7f1a1d7fa700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:09.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 10 mgrc shutdown 2023-01-25T11:41:09.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f1a080367a0 msgr2=0x7f1a08038c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:09.186 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f1a080367a0 0x7f1a08038c50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1a14006fd0 tx=0x7f1a14006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:09.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 msgr2=0x7f1a20104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:09.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20104210 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1a10003f20 tx=0x7f1a10011ae0 comp rx=0 tx=0).stop 2023-01-25T11:41:09.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 shutdown_connections 2023-01-25T11:41:09.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f1a080367a0 0x7f1a08038c50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:09.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 --2- 172.21.15.119:0/4059005205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a201048a0 0x7f1a20104210 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:09.187 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 >> 172.21.15.119:0/4059005205 conn(0x7f1a200fe220 msgr2=0x7f1a200ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:09.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 shutdown_connections 2023-01-25T11:41:09.188 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:08.733+0000 7f1a26750700 1 -- 172.21.15.119:0/4059005205 wait complete. 2023-01-25T11:41:09.188 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying unmanaged mon service... 2023-01-25T11:41:09.288 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:09 smithi119 ceph-mon[102162]: Deploying cephadm binary to smithi119 2023-01-25T11:41:09.288 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:09 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:09.288 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:09 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:10.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:10 smithi119 ceph-mon[102162]: Added host smithi119 2023-01-25T11:41:10.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:10 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:10.214 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-01-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.577+0000 7fba64b6b700 1 Processor -- start 2023-01-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.577+0000 7fba64b6b700 1 -- start start 2023-01-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.577+0000 7fba64b6b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba60106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.577+0000 7fba64b6b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba60107160 con 0x7fba60104840 2023-01-25T11:41:10.215 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba60106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:10.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba60106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54284/0 (socket says 172.21.15.119:54284) 2023-01-25T11:41:10.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5e59c700 1 -- 172.21.15.119:0/682471168 learned_addr learned my addr 172.21.15.119:0/682471168 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:10.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5e59c700 1 -- 172.21.15.119:0/682471168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba601072a0 con 0x7fba60104840 2023-01-25T11:41:10.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5e59c700 1 --2- 172.21.15.119:0/682471168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba60106c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fba500098a0 tx=0x7fba50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dae0034fe688499 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:10.216 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5dd9b700 1 -- 172.21.15.119:0/682471168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba50004160 con 0x7fba60104840 2023-01-25T11:41:10.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5dd9b700 1 -- 172.21.15.119:0/682471168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fba500036a0 con 0x7fba60104840 2023-01-25T11:41:10.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.578+0000 7fba5dd9b700 1 -- 172.21.15.119:0/682471168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba50011870 con 0x7fba60104840 2023-01-25T11:41:10.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.579+0000 7fba64b6b700 1 -- 172.21.15.119:0/682471168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 msgr2=0x7fba60106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:10.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.579+0000 7fba64b6b700 1 --2- 172.21.15.119:0/682471168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba60106c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fba500098a0 tx=0x7fba50009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:10.217 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.579+0000 7fba64b6b700 1 -- 172.21.15.119:0/682471168 shutdown_connections 2023-01-25T11:41:10.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.579+0000 7fba64b6b700 1 --2- 172.21.15.119:0/682471168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba60106c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:10.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.579+0000 7fba64b6b700 1 -- 172.21.15.119:0/682471168 >> 172.21.15.119:0/682471168 conn(0x7fba600fe220 msgr2=0x7fba60100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:10.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.579+0000 7fba64b6b700 1 -- 172.21.15.119:0/682471168 shutdown_connections 2023-01-25T11:41:10.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.579+0000 7fba64b6b700 1 -- 172.21.15.119:0/682471168 wait complete. 2023-01-25T11:41:10.218 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba64b6b700 1 Processor -- start 2023-01-25T11:41:10.219 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba64b6b700 1 -- start start 2023-01-25T11:41:10.219 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba64b6b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba601041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:10.219 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba64b6b700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba601046f0 con 0x7fba60104840 2023-01-25T11:41:10.219 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba5e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba601041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:10.219 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba5e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba601041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54288/0 (socket says 172.21.15.119:54288) 2023-01-25T11:41:10.220 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba5e59c700 1 -- 172.21.15.119:0/2736431056 learned_addr learned my addr 172.21.15.119:0/2736431056 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:10.220 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba5e59c700 1 -- 172.21.15.119:0/2736431056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba50009550 con 0x7fba60104840 2023-01-25T11:41:10.220 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba5e59c700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba601041b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fba500119a0 tx=0x7fba50011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:10.220 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba4f7fe700 1 -- 172.21.15.119:0/2736431056 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba50004110 con 0x7fba60104840 2023-01-25T11:41:10.220 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.580+0000 7fba4f7fe700 1 -- 172.21.15.119:0/2736431056 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7fba5001b440 con 0x7fba60104840 2023-01-25T11:41:10.221 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.581+0000 7fba4f7fe700 1 -- 172.21.15.119:0/2736431056 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba500193d0 con 0x7fba60104840 2023-01-25T11:41:10.221 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.581+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba60108110 con 0x7fba60104840 2023-01-25T11:41:10.221 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.581+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba60107f80 con 0x7fba60104840 2023-01-25T11:41:10.221 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4f7fe700 1 -- 172.21.15.119:0/2736431056 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7fba50017020 con 0x7fba60104840 2023-01-25T11:41:10.221 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4f7fe700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:10.222 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba601082a0 con 0x7fba60104840 2023-01-25T11:41:10.222 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4f7fe700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:10.222 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4f7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:10.222 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4f7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:10.222 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4f7fe700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fba48036750 0x7fba48038c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:10.222 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4f7fe700 1 -- 172.21.15.119:0/2736431056 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fba50049aa0 con 0x7fba60104840 2023-01-25T11:41:10.223 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4ffff700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fba48036750 0x7fba48038c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:10.223 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.582+0000 7fba4ffff700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fba48036750 0x7fba48038c00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba54006fd0 tx=0x7fba54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:10.223 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.587+0000 7fba4f7fe700 1 -- 172.21.15.119:0/2736431056 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fba5001b950 con 0x7fba60104840 2023-01-25T11:41:10.223 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.790+0000 7fba64b6b700 20 mgrc start_command cmd: [{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}] 2023-01-25T11:41:10.223 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.790+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fba60061b00 con 0x7fba48036750 2023-01-25T11:41:10.223 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.794+0000 7fba4f7fe700 1 -- 172.21.15.119:0/2736431056 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fba60061b00 con 0x7fba48036750 2023-01-25T11:41:10.224 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.794+0000 7fba4f7fe700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:10.224 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 10 mgrc shutdown 2023-01-25T11:41:10.224 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fba48036750 msgr2=0x7fba48038c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:10.224 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fba48036750 0x7fba48038c00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fba54006fd0 tx=0x7fba54006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:10.224 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 msgr2=0x7fba601041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:10.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba601041b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fba500119a0 tx=0x7fba50011b30 comp rx=0 tx=0).stop 2023-01-25T11:41:10.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 shutdown_connections 2023-01-25T11:41:10.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fba48036750 0x7fba48038c00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:10.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 --2- 172.21.15.119:0/2736431056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba60104840 0x7fba601041b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:10.225 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 >> 172.21.15.119:0/2736431056 conn(0x7fba600fe220 msgr2=0x7fba600fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:10.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 shutdown_connections 2023-01-25T11:41:10.226 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:09.797+0000 7fba64b6b700 1 -- 172.21.15.119:0/2736431056 wait complete. 2023-01-25T11:41:10.226 INFO:teuthology.orchestra.run.smithi119.stdout:Deploying unmanaged mgr service... 2023-01-25T11:41:11.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:11 smithi119 ceph-mon[102162]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:11.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:11 smithi119 ceph-mon[102162]: Saving service mon spec with placement count:5 2023-01-25T11:41:11.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:11 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:11.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-01-25T11:41:11.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.677+0000 7f2364c3e700 1 Processor -- start 2023-01-25T11:41:11.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.677+0000 7f2364c3e700 1 -- start start 2023-01-25T11:41:11.555 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f2364c3e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f2360106c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:11.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f2364c3e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2360107160 con 0x7f2360104840 2023-01-25T11:41:11.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f2360106c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:11.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f2360106c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54292/0 (socket says 172.21.15.119:54292) 2023-01-25T11:41:11.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235e59c700 1 -- 172.21.15.119:0/245391698 learned_addr learned my addr 172.21.15.119:0/245391698 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:11.556 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235e59c700 1 -- 172.21.15.119:0/245391698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23601072a0 con 0x7f2360104840 2023-01-25T11:41:11.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235e59c700 1 --2- 172.21.15.119:0/245391698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f2360106c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23500098a0 tx=0x7f2350009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11868874eca22 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:11.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235d59a700 1 -- 172.21.15.119:0/245391698 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2350004160 con 0x7f2360104840 2023-01-25T11:41:11.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235d59a700 1 -- 172.21.15.119:0/245391698 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f23500036a0 con 0x7f2360104840 2023-01-25T11:41:11.557 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.678+0000 7f235d59a700 1 -- 172.21.15.119:0/245391698 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2350011870 con 0x7f2360104840 2023-01-25T11:41:11.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.679+0000 7f2364c3e700 1 -- 172.21.15.119:0/245391698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 msgr2=0x7f2360106c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:11.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.679+0000 7f2364c3e700 1 --2- 172.21.15.119:0/245391698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f2360106c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f23500098a0 tx=0x7f2350009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:11.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.679+0000 7f2364c3e700 1 -- 172.21.15.119:0/245391698 shutdown_connections 2023-01-25T11:41:11.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.679+0000 7f2364c3e700 1 --2- 172.21.15.119:0/245391698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f2360106c20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:11.558 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.679+0000 7f2364c3e700 1 -- 172.21.15.119:0/245391698 >> 172.21.15.119:0/245391698 conn(0x7f23600fe220 msgr2=0x7f2360100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:11.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.679+0000 7f2364c3e700 1 -- 172.21.15.119:0/245391698 shutdown_connections 2023-01-25T11:41:11.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.679+0000 7f2364c3e700 1 -- 172.21.15.119:0/245391698 wait complete. 2023-01-25T11:41:11.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f2364c3e700 1 Processor -- start 2023-01-25T11:41:11.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f2364c3e700 1 -- start start 2023-01-25T11:41:11.559 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f2364c3e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f23601041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:11.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f2364c3e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23601046f0 con 0x7f2360104840 2023-01-25T11:41:11.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f235e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f23601041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:11.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f235e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f23601041b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54306/0 (socket says 172.21.15.119:54306) 2023-01-25T11:41:11.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f235e59c700 1 -- 172.21.15.119:0/2691333199 learned_addr learned my addr 172.21.15.119:0/2691333199 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:11.560 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f235e59c700 1 -- 172.21.15.119:0/2691333199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2350009550 con 0x7f2360104840 2023-01-25T11:41:11.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f235e59c700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f23601041b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2350000c00 tx=0x7f2350011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:11.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.680+0000 7f234f7fe700 1 -- 172.21.15.119:0/2691333199 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2350004110 con 0x7f2360104840 2023-01-25T11:41:11.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.681+0000 7f234f7fe700 1 -- 172.21.15.119:0/2691333199 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1004+0+0 (secure 0 0 0) 0x7f235001b440 con 0x7f2360104840 2023-01-25T11:41:11.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.681+0000 7f234f7fe700 1 -- 172.21.15.119:0/2691333199 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f23500193f0 con 0x7f2360104840 2023-01-25T11:41:11.561 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.681+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2360108110 con 0x7f2360104840 2023-01-25T11:41:11.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.681+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2360107f80 con 0x7f2360104840 2023-01-25T11:41:11.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f234f7fe700 1 -- 172.21.15.119:0/2691333199 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7f2350021030 con 0x7f2360104840 2023-01-25T11:41:11.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f234f7fe700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:11.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f234f7fe700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:11.562 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f234f7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:11.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f234f7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:11.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f234f7fe700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f23480367c0 0x7f2348038c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:11.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23601082a0 con 0x7f2360104840 2023-01-25T11:41:11.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f234f7fe700 1 -- 172.21.15.119:0/2691333199 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2350049970 con 0x7f2360104840 2023-01-25T11:41:11.563 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.682+0000 7f235dd9b700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f23480367c0 0x7f2348038c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:11.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.683+0000 7f235dd9b700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f23480367c0 0x7f2348038c70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2354006fd0 tx=0x7f2354006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:11.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.687+0000 7f234f7fe700 1 -- 172.21.15.119:0/2691333199 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f2350022bb0 con 0x7f2360104840 2023-01-25T11:41:11.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.898+0000 7f2364c3e700 20 mgrc start_command cmd: [{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}] 2023-01-25T11:41:11.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.898+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2360061b00 con 0x7f23480367c0 2023-01-25T11:41:11.564 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.903+0000 7f234f7fe700 1 -- 172.21.15.119:0/2691333199 <== mgr.14120 v2:172.21.15.119:6800/569691540 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f2360061b00 con 0x7f23480367c0 2023-01-25T11:41:11.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.903+0000 7f234f7fe700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:11.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.905+0000 7f2364c3e700 10 mgrc shutdown 2023-01-25T11:41:11.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.905+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f23480367c0 msgr2=0x7f2348038c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:11.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.905+0000 7f2364c3e700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f23480367c0 0x7f2348038c70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2354006fd0 tx=0x7f2354006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:11.565 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.905+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 msgr2=0x7f23601041b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:11.566 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.905+0000 7f2364c3e700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f23601041b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2350000c00 tx=0x7f2350011ae0 comp rx=0 tx=0).stop 2023-01-25T11:41:11.566 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.906+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 shutdown_connections 2023-01-25T11:41:11.566 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.906+0000 7f2364c3e700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f23480367c0 0x7f2348038c70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:11.566 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.906+0000 7f2364c3e700 1 --2- 172.21.15.119:0/2691333199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2360104840 0x7f23601041b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:11.566 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.906+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 >> 172.21.15.119:0/2691333199 conn(0x7f23600fe220 msgr2=0x7f23600fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:11.567 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.906+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 shutdown_connections 2023-01-25T11:41:11.567 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:10.906+0000 7f2364c3e700 1 -- 172.21.15.119:0/2691333199 wait complete. 2023-01-25T11:41:12.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:12 smithi119 ceph-mon[102162]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:12.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:12 smithi119 ceph-mon[102162]: Saving service mgr spec with placement count:2 2023-01-25T11:41:12.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:12 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:12.800 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.136+0000 7f242c9ab700 1 Processor -- start 2023-01-25T11:41:12.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.136+0000 7f242c9ab700 1 -- start start 2023-01-25T11:41:12.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.136+0000 7f242c9ab700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:12.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.136+0000 7f242c9ab700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2428077f70 con 0x7f24281048a0 2023-01-25T11:41:12.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:12.801 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54310/0 (socket says 172.21.15.119:54310) 2023-01-25T11:41:12.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242659c700 1 -- 172.21.15.119:0/1197220035 learned_addr learned my addr 172.21.15.119:0/1197220035 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:12.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242659c700 1 -- 172.21.15.119:0/1197220035 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24281071c0 con 0x7f24281048a0 2023-01-25T11:41:12.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242659c700 1 --2- 172.21.15.119:0/1197220035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428106c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24180098a0 tx=0x7f2418009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b2eaa05fc7ef7bd server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:12.802 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242559a700 1 -- 172.21.15.119:0/1197220035 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2418004160 con 0x7f24281048a0 2023-01-25T11:41:12.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242559a700 1 -- 172.21.15.119:0/1197220035 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f24180042c0 con 0x7f24281048a0 2023-01-25T11:41:12.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.137+0000 7f242559a700 1 -- 172.21.15.119:0/1197220035 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2418011830 con 0x7f24281048a0 2023-01-25T11:41:12.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.138+0000 7f242c9ab700 1 -- 172.21.15.119:0/1197220035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 msgr2=0x7f2428106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:12.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.138+0000 7f242c9ab700 1 --2- 172.21.15.119:0/1197220035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428106c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f24180098a0 tx=0x7f2418009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:12.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.138+0000 7f242c9ab700 1 -- 172.21.15.119:0/1197220035 shutdown_connections 2023-01-25T11:41:12.803 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.138+0000 7f242c9ab700 1 --2- 172.21.15.119:0/1197220035 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428106c80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:12.804 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.138+0000 7f242c9ab700 1 -- 172.21.15.119:0/1197220035 >> 172.21.15.119:0/1197220035 conn(0x7f24280fe220 msgr2=0x7f2428100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:12.804 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.138+0000 7f242c9ab700 1 -- 172.21.15.119:0/1197220035 shutdown_connections 2023-01-25T11:41:12.804 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.138+0000 7f242c9ab700 1 -- 172.21.15.119:0/1197220035 wait complete. 2023-01-25T11:41:12.804 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.139+0000 7f242c9ab700 1 Processor -- start 2023-01-25T11:41:12.804 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.139+0000 7f242c9ab700 1 -- start start 2023-01-25T11:41:12.805 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.139+0000 7f242c9ab700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:12.805 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.139+0000 7f242c9ab700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24281980b0 con 0x7f24281048a0 2023-01-25T11:41:12.805 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.139+0000 7f242659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:12.805 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.139+0000 7f242659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54318/0 (socket says 172.21.15.119:54318) 2023-01-25T11:41:12.805 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.139+0000 7f242659c700 1 -- 172.21.15.119:0/2041063224 learned_addr learned my addr 172.21.15.119:0/2041063224 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:12.805 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.140+0000 7f242659c700 1 -- 172.21.15.119:0/2041063224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418009550 con 0x7f24281048a0 2023-01-25T11:41:12.806 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.140+0000 7f242659c700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428197b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2418009870 tx=0x7f2418011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:12.806 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.140+0000 7f24177fe700 1 -- 172.21.15.119:0/2041063224 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24180036a0 con 0x7f24281048a0 2023-01-25T11:41:12.806 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.140+0000 7f24177fe700 1 -- 172.21.15.119:0/2041063224 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f2418003800 con 0x7f24281048a0 2023-01-25T11:41:12.806 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.140+0000 7f24177fe700 1 -- 172.21.15.119:0/2041063224 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24180193c0 con 0x7f24281048a0 2023-01-25T11:41:12.806 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.141+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2428108190 con 0x7f24281048a0 2023-01-25T11:41:12.807 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.141+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2428108000 con 0x7f24281048a0 2023-01-25T11:41:12.807 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2428198af0 con 0x7f24281048a0 2023-01-25T11:41:12.807 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f24177fe700 1 -- 172.21.15.119:0/2041063224 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7f2418017020 con 0x7f24281048a0 2023-01-25T11:41:12.807 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f24177fe700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:12.807 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f24177fe700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:12.807 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f24177fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f24177fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f24177fe700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f2410036770 0x7f2410038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f24177fe700 1 -- 172.21.15.119:0/2041063224 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2418049b80 con 0x7f24281048a0 2023-01-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.142+0000 7f2425d9b700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f2410036770 0x7f2410038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:12.808 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.143+0000 7f2425d9b700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f2410036770 0x7f2410038c20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f241c006fd0 tx=0x7f241c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:12.809 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.147+0000 7f24177fe700 1 -- 172.21.15.119:0/2041063224 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f2418018db0 con 0x7f24281048a0 2023-01-25T11:41:12.809 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.338+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f2428108320 con 0x7f24281048a0 2023-01-25T11:41:12.809 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.341+0000 7f24177fe700 1 -- 172.21.15.119:0/2041063224 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f2418018350 con 0x7f24281048a0 2023-01-25T11:41:12.809 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 10 mgrc shutdown 2023-01-25T11:41:12.809 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f2410036770 msgr2=0x7f2410038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f2410036770 0x7f2410038c20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f241c006fd0 tx=0x7f241c006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 msgr2=0x7f2428197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428197b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2418009870 tx=0x7f2418011910 comp rx=0 tx=0).stop 2023-01-25T11:41:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 shutdown_connections 2023-01-25T11:41:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f2410036770 0x7f2410038c20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:12.810 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 --2- 172.21.15.119:0/2041063224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24281048a0 0x7f2428197b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 >> 172.21.15.119:0/2041063224 conn(0x7f24280fe220 msgr2=0x7f24280ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 shutdown_connections 2023-01-25T11:41:12.811 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:12.344+0000 7f242c9ab700 1 -- 172.21.15.119:0/2041063224 wait complete. 2023-01-25T11:41:13.640 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:13 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:13.640 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:13 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/2041063224' entity='client.admin' 2023-01-25T11:41:14.143 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf3ae0f700 1 Processor -- start 2023-01-25T11:41:14.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf3ae0f700 1 -- start start 2023-01-25T11:41:14.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf3ae0f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf341047b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:14.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf3ae0f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34104d80 con 0x7fcf341043a0 2023-01-25T11:41:14.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf38bab700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf341047b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:14.144 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf38bab700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf341047b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54328/0 (socket says 172.21.15.119:54328) 2023-01-25T11:41:14.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf38bab700 1 -- 172.21.15.119:0/1303982486 learned_addr learned my addr 172.21.15.119:0/1303982486 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:14.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.297+0000 7fcf38bab700 1 -- 172.21.15.119:0/1303982486 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34102e50 con 0x7fcf341043a0 2023-01-25T11:41:14.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.298+0000 7fcf38bab700 1 --2- 172.21.15.119:0/1303982486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf341047b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf280098a0 tx=0x7fcf28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156824107a8f35e1 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:14.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.298+0000 7fcf337fe700 1 -- 172.21.15.119:0/1303982486 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf28004160 con 0x7fcf341043a0 2023-01-25T11:41:14.145 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.298+0000 7fcf337fe700 1 -- 172.21.15.119:0/1303982486 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fcf280042c0 con 0x7fcf341043a0 2023-01-25T11:41:14.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.298+0000 7fcf337fe700 1 -- 172.21.15.119:0/1303982486 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf28011830 con 0x7fcf341043a0 2023-01-25T11:41:14.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.299+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/1303982486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 msgr2=0x7fcf341047b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:14.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.299+0000 7fcf3ae0f700 1 --2- 172.21.15.119:0/1303982486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf341047b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcf280098a0 tx=0x7fcf28009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:14.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.299+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/1303982486 shutdown_connections 2023-01-25T11:41:14.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.299+0000 7fcf3ae0f700 1 --2- 172.21.15.119:0/1303982486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf341047b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:14.146 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.299+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/1303982486 >> 172.21.15.119:0/1303982486 conn(0x7fcf340fd9b0 msgr2=0x7fcf340ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:14.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.299+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/1303982486 shutdown_connections 2023-01-25T11:41:14.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.299+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/1303982486 wait complete. 2023-01-25T11:41:14.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf3ae0f700 1 Processor -- start 2023-01-25T11:41:14.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf3ae0f700 1 -- start start 2023-01-25T11:41:14.147 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf3ae0f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf34199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:14.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf3ae0f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3419a270 con 0x7fcf341043a0 2023-01-25T11:41:14.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf38bab700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf34199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:14.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf38bab700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf34199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54342/0 (socket says 172.21.15.119:54342) 2023-01-25T11:41:14.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf38bab700 1 -- 172.21.15.119:0/2060510047 learned_addr learned my addr 172.21.15.119:0/2060510047 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:14.148 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf38bab700 1 -- 172.21.15.119:0/2060510047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28009550 con 0x7fcf341043a0 2023-01-25T11:41:14.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.300+0000 7fcf38bab700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf34199d30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf28011860 tx=0x7fcf28011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:14.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.301+0000 7fcf31ffb700 1 -- 172.21.15.119:0/2060510047 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf28011e10 con 0x7fcf341043a0 2023-01-25T11:41:14.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.301+0000 7fcf31ffb700 1 -- 172.21.15.119:0/2060510047 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fcf280038b0 con 0x7fcf341043a0 2023-01-25T11:41:14.149 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.301+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf34101d40 con 0x7fcf341043a0 2023-01-25T11:41:14.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.301+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf34103c40 con 0x7fcf341043a0 2023-01-25T11:41:14.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.301+0000 7fcf31ffb700 1 -- 172.21.15.119:0/2060510047 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf2801ae70 con 0x7fcf341043a0 2023-01-25T11:41:14.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf31ffb700 1 -- 172.21.15.119:0/2060510047 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7fcf2802a020 con 0x7fcf341043a0 2023-01-25T11:41:14.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf31ffb700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:14.150 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf31ffb700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:14.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf31ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:14.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf31ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:14.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf31ffb700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fcf240367a0 0x7fcf24038c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:14.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf31ffb700 1 -- 172.21.15.119:0/2060510047 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcf2804a810 con 0x7fcf341043a0 2023-01-25T11:41:14.151 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf18005110 con 0x7fcf341043a0 2023-01-25T11:41:14.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf33fff700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fcf240367a0 0x7fcf24038c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:14.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.302+0000 7fcf33fff700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fcf240367a0 0x7fcf24038c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006fd0 tx=0x7fcf20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:14.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.310+0000 7fcf31ffb700 1 -- 172.21.15.119:0/2060510047 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fcf28018940 con 0x7fcf341043a0 2023-01-25T11:41:14.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.503+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fcf18005d60 con 0x7fcf341043a0 2023-01-25T11:41:14.152 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.506+0000 7fcf31ffb700 1 -- 172.21.15.119:0/2060510047 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fcf28018350 con 0x7fcf341043a0 2023-01-25T11:41:14.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 10 mgrc shutdown 2023-01-25T11:41:14.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fcf240367a0 msgr2=0x7fcf24038c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:14.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fcf240367a0 0x7fcf24038c50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf20006fd0 tx=0x7fcf20006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:14.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 msgr2=0x7fcf34199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:14.153 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf34199d30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcf28011860 tx=0x7fcf28011940 comp rx=0 tx=0).stop 2023-01-25T11:41:14.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 shutdown_connections 2023-01-25T11:41:14.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fcf240367a0 0x7fcf24038c50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:14.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 --2- 172.21.15.119:0/2060510047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf341043a0 0x7fcf34199d30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:14.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 >> 172.21.15.119:0/2060510047 conn(0x7fcf340fd9b0 msgr2=0x7fcf3418c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:14.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 shutdown_connections 2023-01-25T11:41:14.154 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:13.508+0000 7fcf3ae0f700 1 -- 172.21.15.119:0/2060510047 wait complete. 2023-01-25T11:41:14.155 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling the dashboard module... 2023-01-25T11:41:14.640 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:14 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/2060510047' entity='client.admin' 2023-01-25T11:41:14.640 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:14 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.544+0000 7f8130232700 1 Processor -- start 2023-01-25T11:41:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f8130232700 1 -- start start 2023-01-25T11:41:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f8130232700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128103f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:15.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f8130232700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128104530 con 0x7f8128103b50 2023-01-25T11:41:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f812dfce700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128103f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f812dfce700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128103f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54358/0 (socket says 172.21.15.119:54358) 2023-01-25T11:41:15.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f812dfce700 1 -- 172.21.15.119:0/1151867545 learned_addr learned my addr 172.21.15.119:0/1151867545 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f812dfce700 1 -- 172.21.15.119:0/1151867545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8128102080 con 0x7f8128103b50 2023-01-25T11:41:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.545+0000 7f812dfce700 1 --2- 172.21.15.119:0/1151867545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128103f60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81240098a0 tx=0x7f8124009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb7830a3b1a4c8e5 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f812cfcc700 1 -- 172.21.15.119:0/1151867545 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8124004160 con 0x7f8128103b50 2023-01-25T11:41:15.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f812cfcc700 1 -- 172.21.15.119:0/1151867545 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f81240042c0 con 0x7f8128103b50 2023-01-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f812cfcc700 1 -- 172.21.15.119:0/1151867545 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8124011830 con 0x7f8128103b50 2023-01-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f8130232700 1 -- 172.21.15.119:0/1151867545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 msgr2=0x7f8128103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f8130232700 1 --2- 172.21.15.119:0/1151867545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128103f60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81240098a0 tx=0x7f8124009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:15.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f8130232700 1 -- 172.21.15.119:0/1151867545 shutdown_connections 2023-01-25T11:41:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f8130232700 1 --2- 172.21.15.119:0/1151867545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128103f60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f8130232700 1 -- 172.21.15.119:0/1151867545 >> 172.21.15.119:0/1151867545 conn(0x7f81280fd9b0 msgr2=0x7f81280ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f8130232700 1 -- 172.21.15.119:0/1151867545 shutdown_connections 2023-01-25T11:41:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.546+0000 7f8130232700 1 -- 172.21.15.119:0/1151867545 wait complete. 2023-01-25T11:41:15.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f8130232700 1 Processor -- start 2023-01-25T11:41:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f8130232700 1 -- start start 2023-01-25T11:41:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f8130232700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f8130232700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8128198240 con 0x7f8128103b50 2023-01-25T11:41:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f812dfce700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f812dfce700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54374/0 (socket says 172.21.15.119:54374) 2023-01-25T11:41:15.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f812dfce700 1 -- 172.21.15.119:0/3576159503 learned_addr learned my addr 172.21.15.119:0/3576159503 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.547+0000 7f812dfce700 1 -- 172.21.15.119:0/3576159503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8124009550 con 0x7f8128103b50 2023-01-25T11:41:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.548+0000 7f812dfce700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128197d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8124006ba0 tx=0x7f8124011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.548+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8124011e40 con 0x7f8128103b50 2023-01-25T11:41:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.548+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f812401a430 con 0x7f8128103b50 2023-01-25T11:41:15.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.548+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81281027c0 con 0x7f8128103b50 2023-01-25T11:41:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.548+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f81240193c0 con 0x7f8128103b50 2023-01-25T11:41:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.548+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8128102630 con 0x7f8128103b50 2023-01-25T11:41:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 8) v1 ==== 43509+0+0 (secure 0 0 0) 0x7f812402a020 con 0x7f8128103b50 2023-01-25T11:41:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f811effd700 20 mgrc handle_mgr_map mgrmap(e 8) v1 2023-01-25T11:41:15.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8128198b00 con 0x7f8128103b50 2023-01-25T11:41:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f811effd700 4 mgrc handle_mgr_map Got map version 8 2023-01-25T11:41:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f811effd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f811effd700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:15.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f811effd700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f8114036740 0x7f8114038bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8124049a10 con 0x7f8128103b50 2023-01-25T11:41:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.549+0000 7f812d7cd700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f8114036740 0x7f8114038bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.550+0000 7f812d7cd700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f8114036740 0x7f8114038bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8118006fd0 tx=0x7f8118006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.554+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f812401a8d0 con 0x7f8128103b50 2023-01-25T11:41:15.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:14.798+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8128102950 con 0x7f8128103b50 2023-01-25T11:41:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.512+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 9) v1 ==== 43522+0+0 (secure 0 0 0) 0x7f8124017020 con 0x7f8128103b50 2023-01-25T11:41:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.512+0000 7f811effd700 20 mgrc handle_mgr_map mgrmap(e 9) v1 2023-01-25T11:41:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.512+0000 7f811effd700 4 mgrc handle_mgr_map Got map version 9 2023-01-25T11:41:15.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.512+0000 7f811effd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.513+0000 7f811effd700 1 -- 172.21.15.119:0/3576159503 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8124023bc0 con 0x7f8128103b50 2023-01-25T11:41:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.515+0000 7f8130232700 10 mgrc shutdown 2023-01-25T11:41:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.515+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f8114036740 msgr2=0x7f8114038bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.515+0000 7f8130232700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f8114036740 0x7f8114038bf0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8118006fd0 tx=0x7f8118006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:15.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.515+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 msgr2=0x7f8128197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.515+0000 7f8130232700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128197d00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8124006ba0 tx=0x7f8124011910 comp rx=0 tx=0).stop 2023-01-25T11:41:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.516+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 shutdown_connections 2023-01-25T11:41:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.516+0000 7f8130232700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f8114036740 0x7f8114038bf0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:15.896 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.516+0000 7f8130232700 1 --2- 172.21.15.119:0/3576159503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8128103b50 0x7f8128197d00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.516+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 >> 172.21.15.119:0/3576159503 conn(0x7f81280fd9b0 msgr2=0x7f812818c010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.516+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 shutdown_connections 2023-01-25T11:41:15.897 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:15.516+0000 7f8130232700 1 -- 172.21.15.119:0/3576159503 wait complete. 2023-01-25T11:41:15.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:15 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/3576159503' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-01-25T11:41:15.898 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:15 smithi119 ceph-mon[102162]: from='mgr.14120 172.21.15.119:0/3575628813' entity='mgr.a' 2023-01-25T11:41:15.898 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:15 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: ignoring --setuser ceph since I am not root 2023-01-25T11:41:15.899 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:15 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: ignoring --setgroup ceph since I am not root 2023-01-25T11:41:16.890 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:16 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/3576159503' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-01-25T11:41:16.890 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:16 smithi119 ceph-mon[102162]: mgrmap e9: a(active, since 15s) 2023-01-25T11:41:16.891 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:16 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:16.438+0000 7f5b086e2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T11:41:16.923 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:41:16.923 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-01-25T11:41:16.923 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "available": true, 2023-01-25T11:41:16.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "active_name": "a", 2023-01-25T11:41:16.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-01-25T11:41:16.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:41:16.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.265+0000 7f9b786f4700 1 Processor -- start 2023-01-25T11:41:16.924 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.265+0000 7f9b786f4700 1 -- start start 2023-01-25T11:41:16.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.265+0000 7f9b786f4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70106c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:16.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.265+0000 7f9b786f4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b70077f70 con 0x7f9b701048a0 2023-01-25T11:41:16.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b76490700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70106c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:16.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b76490700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70106c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50508/0 (socket says 172.21.15.119:50508) 2023-01-25T11:41:16.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b76490700 1 -- 172.21.15.119:0/162739827 learned_addr learned my addr 172.21.15.119:0/162739827 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:16.925 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b76490700 1 -- 172.21.15.119:0/162739827 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b701071c0 con 0x7f9b701048a0 2023-01-25T11:41:16.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b76490700 1 --2- 172.21.15.119:0/162739827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70106c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0098a0 tx=0x7f9b6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ce28a5cc661099 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:16.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b7548e700 1 -- 172.21.15.119:0/162739827 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c004160 con 0x7f9b701048a0 2023-01-25T11:41:16.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b7548e700 1 -- 172.21.15.119:0/162739827 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f9b6c0042c0 con 0x7f9b701048a0 2023-01-25T11:41:16.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.266+0000 7f9b7548e700 1 -- 172.21.15.119:0/162739827 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c011830 con 0x7f9b701048a0 2023-01-25T11:41:16.926 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.267+0000 7f9b786f4700 1 -- 172.21.15.119:0/162739827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 msgr2=0x7f9b70106c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:16.927 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.267+0000 7f9b786f4700 1 --2- 172.21.15.119:0/162739827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70106c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c0098a0 tx=0x7f9b6c009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:16.927 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.267+0000 7f9b786f4700 1 -- 172.21.15.119:0/162739827 shutdown_connections 2023-01-25T11:41:16.927 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.267+0000 7f9b786f4700 1 --2- 172.21.15.119:0/162739827 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70106c80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:16.927 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.267+0000 7f9b786f4700 1 -- 172.21.15.119:0/162739827 >> 172.21.15.119:0/162739827 conn(0x7f9b700fe220 msgr2=0x7f9b70100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:16.927 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.267+0000 7f9b786f4700 1 -- 172.21.15.119:0/162739827 shutdown_connections 2023-01-25T11:41:16.928 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.267+0000 7f9b786f4700 1 -- 172.21.15.119:0/162739827 wait complete. 2023-01-25T11:41:16.928 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.268+0000 7f9b786f4700 1 Processor -- start 2023-01-25T11:41:16.928 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.268+0000 7f9b786f4700 1 -- start start 2023-01-25T11:41:16.928 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.268+0000 7f9b786f4700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:16.928 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.268+0000 7f9b786f4700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b701980a0 con 0x7f9b701048a0 2023-01-25T11:41:16.929 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.268+0000 7f9b76490700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:16.929 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.268+0000 7f9b76490700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50516/0 (socket says 172.21.15.119:50516) 2023-01-25T11:41:16.929 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.268+0000 7f9b76490700 1 -- 172.21.15.119:0/3693783494 learned_addr learned my addr 172.21.15.119:0/3693783494 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:16.929 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.269+0000 7f9b76490700 1 -- 172.21.15.119:0/3693783494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b6c009550 con 0x7f9b701048a0 2023-01-25T11:41:16.929 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.269+0000 7f9b76490700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70197b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009870 tx=0x7f9b6c011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:16.930 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.269+0000 7f9b677fe700 1 -- 172.21.15.119:0/3693783494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c0036a0 con 0x7f9b701048a0 2023-01-25T11:41:16.930 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.269+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b70108190 con 0x7f9b701048a0 2023-01-25T11:41:16.930 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.269+0000 7f9b677fe700 1 -- 172.21.15.119:0/3693783494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f9b6c003800 con 0x7f9b701048a0 2023-01-25T11:41:16.930 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.269+0000 7f9b677fe700 1 -- 172.21.15.119:0/3693783494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b6c0193c0 con 0x7f9b701048a0 2023-01-25T11:41:16.930 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.269+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b70108000 con 0x7f9b701048a0 2023-01-25T11:41:16.931 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b677fe700 1 -- 172.21.15.119:0/3693783494 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 43522+0+0 (secure 0 0 0) 0x7f9b6c017020 con 0x7f9b701048a0 2023-01-25T11:41:16.931 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b677fe700 20 mgrc handle_mgr_map mgrmap(e 9) v1 2023-01-25T11:41:16.931 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b677fe700 4 mgrc handle_mgr_map Got map version 9 2023-01-25T11:41:16.931 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b677fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:16.931 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b677fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:16.932 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b677fe700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 0x7f9b5c038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:16.932 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b677fe700 1 -- 172.21.15.119:0/3693783494 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9b6c01a840 con 0x7f9b701048a0 2023-01-25T11:41:16.932 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b75c8f700 1 -- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 msgr2=0x7f9b5c038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/569691540 2023-01-25T11:41:16.932 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b75c8f700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 0x7f9b5c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T11:41:16.932 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.270+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b70198ae0 con 0x7f9b701048a0 2023-01-25T11:41:16.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.275+0000 7f9b677fe700 1 -- 172.21.15.119:0/3693783494 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f9b6c019650 con 0x7f9b701048a0 2023-01-25T11:41:16.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.471+0000 7f9b75c8f700 1 -- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 msgr2=0x7f9b5c038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/569691540 2023-01-25T11:41:16.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.471+0000 7f9b75c8f700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 0x7f9b5c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T11:41:16.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.520+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f9b70108320 con 0x7f9b701048a0 2023-01-25T11:41:16.933 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.520+0000 7f9b677fe700 1 -- 172.21.15.119:0/3693783494 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f9b6c022310 con 0x7f9b701048a0 2023-01-25T11:41:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 10 mgrc shutdown 2023-01-25T11:41:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 msgr2=0x7f9b5c038c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-01-25T11:41:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 0x7f9b5c038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 msgr2=0x7f9b70197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70197b60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c009870 tx=0x7f9b6c011910 comp rx=0 tx=0).stop 2023-01-25T11:41:16.934 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 shutdown_connections 2023-01-25T11:41:16.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7f9b5c036770 0x7f9b5c038c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:16.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 --2- 172.21.15.119:0/3693783494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b701048a0 0x7f9b70197b60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:16.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 >> 172.21.15.119:0/3693783494 conn(0x7f9b700fe220 msgr2=0x7f9b700ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:16.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 shutdown_connections 2023-01-25T11:41:16.935 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:16.523+0000 7f9b786f4700 1 -- 172.21.15.119:0/3693783494 wait complete. 2023-01-25T11:41:16.936 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for the mgr to restart... 2023-01-25T11:41:16.936 INFO:teuthology.orchestra.run.smithi119.stdout:Waiting for mgr epoch 9... 2023-01-25T11:41:17.290 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:17 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:17.013+0000 7f5b086e2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T11:41:17.290 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:17 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:17.106+0000 7f5b086e2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T11:41:17.640 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:17 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/3693783494' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-01-25T11:41:17.641 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:17 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:17.287+0000 7f5b086e2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T11:41:18.390 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:18 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:18.079+0000 7f5b086e2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T11:41:18.391 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:18 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:18.258+0000 7f5b086e2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T11:41:18.766 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:18 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:18.508+0000 7f5b086e2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T11:41:18.766 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:18 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:18.594+0000 7f5b086e2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T11:41:19.048 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:18 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:18.763+0000 7f5b086e2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T11:41:19.390 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:19 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:19.046+0000 7f5b086e2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T11:41:19.390 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:19 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:19.135+0000 7f5b086e2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T11:41:19.733 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:19 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:19.430+0000 7f5b086e2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T11:41:19.734 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:19 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:19.559+0000 7f5b086e2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T11:41:20.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:19 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:19.731+0000 7f5b086e2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T11:41:20.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:19 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:19.824+0000 7f5b086e2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T11:41:20.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:19 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:19.917+0000 7f5b086e2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T11:41:20.640 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:20 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:20.179+0000 7f5b086e2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T11:41:21.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:20 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:20.867+0000 7f5b086e2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T11:41:21.140 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:20 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:20.963+0000 7f5b086e2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T11:41:21.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:21.050+0000 7f5b086e2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T11:41:21.564 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:21.148+0000 7f5b086e2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T11:41:21.831 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:21.562+0000 7f5b086e2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T11:41:21.831 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:21.648+0000 7f5b086e2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T11:41:22.140 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-mon[102162]: Active manager daemon a restarted 2023-01-25T11:41:22.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-mon[102162]: Activating manager daemon a 2023-01-25T11:41:22.141 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-mon[102162]: osdmap e3: 0 total, 0 up, 0 in 2023-01-25T11:41:22.141 INFO:journalctl@ceph.mgr.a.smithi119.stdout:Jan 25 11:41:21 smithi119 ceph-f64523de-9ca4-11ed-9e56-001a4aab830c-mgr-a[102385]: 2023-01-25T11:41:21.828+0000 7f5b086e2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T11:41:23.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: mgrmap e10: a(active, starting, since 0.0526317s) 2023-01-25T11:41:23.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-01-25T11:41:23.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix": "mgr metadata", "who": "a", "id": "a"}]: dispatch 2023-01-25T11:41:23.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-01-25T11:41:23.391 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-01-25T11:41:23.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-01-25T11:41:23.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: Manager daemon a is now available 2023-01-25T11:41:23.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:23.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-01-25T11:41:23.392 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/a/mirror_snapshot_schedule"}]: dispatch 2023-01-25T11:41:23.393 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:22 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/a/trash_purge_schedule"}]: dispatch 2023-01-25T11:41:23.878 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout { 2023-01-25T11:41:23.878 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-01-25T11:41:23.879 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout "initialized": true 2023-01-25T11:41:23.879 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout } 2023-01-25T11:41:23.879 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.293+0000 7fd072f7f700 1 Processor -- start 2023-01-25T11:41:23.879 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd072f7f700 1 -- start start 2023-01-25T11:41:23.879 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd072f7f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:23.880 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd072f7f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c077e50 con 0x7fd06c1026b0 2023-01-25T11:41:23.880 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd070d1b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:23.880 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd070d1b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50526/0 (socket says 172.21.15.119:50526) 2023-01-25T11:41:23.880 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd070d1b700 1 -- 172.21.15.119:0/196799748 learned_addr learned my addr 172.21.15.119:0/196799748 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:23.880 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd070d1b700 1 -- 172.21.15.119:0/196799748 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c103000 con 0x7fd06c1026b0 2023-01-25T11:41:23.881 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd070d1b700 1 --2- 172.21.15.119:0/196799748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c102ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098a0 tx=0x7fd068009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4c1ff1ac5bdab9 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:23.881 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.294+0000 7fd0637fe700 1 -- 172.21.15.119:0/196799748 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd068004160 con 0x7fd06c1026b0 2023-01-25T11:41:23.881 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd0637fe700 1 -- 172.21.15.119:0/196799748 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fd0680042c0 con 0x7fd06c1026b0 2023-01-25T11:41:23.881 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd0637fe700 1 -- 172.21.15.119:0/196799748 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd068011830 con 0x7fd06c1026b0 2023-01-25T11:41:23.881 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd072f7f700 1 -- 172.21.15.119:0/196799748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 msgr2=0x7fd06c102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:23.882 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd072f7f700 1 --2- 172.21.15.119:0/196799748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c102ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd0680098a0 tx=0x7fd068009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:23.882 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd072f7f700 1 -- 172.21.15.119:0/196799748 shutdown_connections 2023-01-25T11:41:23.882 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd072f7f700 1 --2- 172.21.15.119:0/196799748 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c102ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:23.882 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd072f7f700 1 -- 172.21.15.119:0/196799748 >> 172.21.15.119:0/196799748 conn(0x7fd06c0fe220 msgr2=0x7fd06c100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:23.882 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd072f7f700 1 -- 172.21.15.119:0/196799748 shutdown_connections 2023-01-25T11:41:23.882 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.295+0000 7fd072f7f700 1 -- 172.21.15.119:0/196799748 wait complete. 2023-01-25T11:41:23.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd072f7f700 1 Processor -- start 2023-01-25T11:41:23.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd072f7f700 1 -- start start 2023-01-25T11:41:23.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd072f7f700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:23.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd072f7f700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c19a190 con 0x7fd06c1026b0 2023-01-25T11:41:23.883 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd070d1b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:23.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd070d1b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c199c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50530/0 (socket says 172.21.15.119:50530) 2023-01-25T11:41:23.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd070d1b700 1 -- 172.21.15.119:0/1039877520 learned_addr learned my addr 172.21.15.119:0/1039877520 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:23.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.296+0000 7fd070d1b700 1 -- 172.21.15.119:0/1039877520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068009550 con 0x7fd06c1026b0 2023-01-25T11:41:23.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.297+0000 7fd070d1b700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c199c50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd068009290 tx=0x7fd068011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:23.884 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.297+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd068011e40 con 0x7fd06c1026b0 2023-01-25T11:41:23.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.297+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c103f80 con 0x7fd06c1026b0 2023-01-25T11:41:23.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.297+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fd068024440 con 0x7fd06c1026b0 2023-01-25T11:41:23.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.297+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd06801a430 con 0x7fd06c1026b0 2023-01-25T11:41:23.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.297+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c103df0 con 0x7fd06c1026b0 2023-01-25T11:41:23.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 43522+0+0 (secure 0 0 0) 0x7fd068017020 con 0x7fd06c1026b0 2023-01-25T11:41:23.885 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd061ffb700 20 mgrc handle_mgr_map mgrmap(e 9) v1 2023-01-25T11:41:23.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd061ffb700 4 mgrc handle_mgr_map Got map version 9 2023-01-25T11:41:23.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd061ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:23.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd061ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] 2023-01-25T11:41:23.886 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd061ffb700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd072f7f700 20 mgrc start_tell_command target: cmd: [{"prefix": "get_command_descriptions"}] 2023-01-25T11:41:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 --> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd06c104110 con 0x7fd054036770 2023-01-25T11:41:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd06804aa00 con 0x7fd06c1026b0 2023-01-25T11:41:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd063fff700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 msgr2=0x7fd054038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/569691540 2023-01-25T11:41:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.298+0000 7fd063fff700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-01-25T11:41:23.887 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.499+0000 7fd063fff700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 msgr2=0x7fd054038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/569691540 2023-01-25T11:41:23.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.499+0000 7fd063fff700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-01-25T11:41:23.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.900+0000 7fd063fff700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 msgr2=0x7fd054038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/569691540 2023-01-25T11:41:23.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:17.900+0000 7fd063fff700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-01-25T11:41:23.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:18.701+0000 7fd063fff700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 msgr2=0x7fd054038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/569691540 2023-01-25T11:41:23.888 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:18.701+0000 7fd063fff700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-01-25T11:41:23.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:20.302+0000 7fd063fff700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 msgr2=0x7fd054038c20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6800/569691540 2023-01-25T11:41:23.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:20.302+0000 7fd063fff700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-01-25T11:41:23.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.893+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mgrmap(e 10) v1 ==== 43332+0+0 (secure 0 0 0) 0x7fd06800f050 con 0x7fd06c1026b0 2023-01-25T11:41:23.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.893+0000 7fd061ffb700 20 mgrc handle_mgr_map mgrmap(e 10) v1 2023-01-25T11:41:23.889 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.894+0000 7fd061ffb700 4 mgrc handle_mgr_map Got map version 10 2023-01-25T11:41:23.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.894+0000 7fd061ffb700 4 mgrc handle_mgr_map Active mgr is now 2023-01-25T11:41:23.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.894+0000 7fd061ffb700 4 mgrc reconnect Terminating session with v2:172.21.15.119:6800/569691540 2023-01-25T11:41:23.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.894+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 msgr2=0x7fd054038c20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-01-25T11:41:23.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.894+0000 7fd061ffb700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:23.890 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:21.894+0000 7fd061ffb700 4 mgrc reconnect No active mgr available yet 2023-01-25T11:41:23.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 11) v1 ==== 43442+0+0 (secure 0 0 0) 0x7fd068044d20 con 0x7fd06c1026b0 2023-01-25T11:41:23.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 20 mgrc handle_mgr_map mgrmap(e 11) v1 2023-01-25T11:41:23.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 4 mgrc handle_mgr_map Got map version 11 2023-01-25T11:41:23.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:23.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:23.891 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fd054039fe0 0x7fd05403c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:23.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 10 mgrc reconnect resending 0 (tell) 2023-01-25T11:41:23.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 --> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd06c104110 con 0x7fd054039fe0 2023-01-25T11:41:23.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.901+0000 7fd063fff700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fd054039fe0 0x7fd05403c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:23.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.902+0000 7fd063fff700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fd054039fe0 0x7fd05403c3c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd05c003a10 tx=0x7fd05c0092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:23.892 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.902+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mgr.14152 v2:172.21.15.119:6800/4087238518 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7fd06c104110 con 0x7fd054039fe0 2023-01-25T11:41:23.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.902+0000 7fd061ffb700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:23.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.908+0000 7fd072f7f700 20 mgrc start_tell_command target: cmd: [{"prefix": "mgr_status"}] 2023-01-25T11:41:23.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.908+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 --> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd06c061b00 con 0x7fd054039fe0 2023-01-25T11:41:23.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd061ffb700 1 -- 172.21.15.119:0/1039877520 <== mgr.14152 v2:172.21.15.119:6800/4087238518 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd06c061b00 con 0x7fd054039fe0 2023-01-25T11:41:23.893 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd061ffb700 20 mgrc handle_command_reply tid 1 r 0 2023-01-25T11:41:23.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 10 mgrc shutdown 2023-01-25T11:41:23.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fd054039fe0 msgr2=0x7fd05403c3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:23.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fd054039fe0 0x7fd05403c3c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd05c003a10 tx=0x7fd05c0092b0 comp rx=0 tx=0).stop 2023-01-25T11:41:23.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 msgr2=0x7fd06c199c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:23.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c199c50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd068009290 tx=0x7fd068011910 comp rx=0 tx=0).stop 2023-01-25T11:41:23.894 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 shutdown_connections 2023-01-25T11:41:23.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fd054039fe0 0x7fd05403c3c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:23.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd06c1026b0 0x7fd06c199c50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:23.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 --2- 172.21.15.119:0/1039877520 >> [v2:172.21.15.119:6800/569691540,v1:172.21.15.119:6801/569691540] conn(0x7fd054036770 0x7fd054038c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:23.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 >> 172.21.15.119:0/1039877520 conn(0x7fd06c0fe220 msgr2=0x7fd06c18e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:23.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 shutdown_connections 2023-01-25T11:41:23.895 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:22.909+0000 7fd072f7f700 1 -- 172.21.15.119:0/1039877520 wait complete. 2023-01-25T11:41:23.896 INFO:teuthology.orchestra.run.smithi119.stdout:mgr epoch 9 is available 2023-01-25T11:41:23.896 INFO:teuthology.orchestra.run.smithi119.stdout:Generating a dashboard self-signed certificate... 2023-01-25T11:41:24.163 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:23 smithi119 ceph-mon[102162]: mgrmap e11: a(active, since 1.06074s) 2023-01-25T11:41:24.164 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:23 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:23] ENGINE Bus STARTING 2023-01-25T11:41:24.164 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:23 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:23] ENGINE Serving on https://172.21.15.119:8765 2023-01-25T11:41:24.164 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:23 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:23] ENGINE Serving on https://172.21.15.119:7150 2023-01-25T11:41:24.164 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:23 smithi119 ceph-mon[102162]: [25/Jan/2023:11:41:23] ENGINE Bus STARTED 2023-01-25T11:41:25.071 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:24 smithi119 ceph-mon[102162]: mgrmap e12: a(active, since 2s) 2023-01-25T11:41:25.071 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:24 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' 2023-01-25T11:41:25.072 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:24 smithi119 ceph-mon[102162]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:25.072 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:24 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' 2023-01-25T11:41:25.072 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:24 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' 2023-01-25T11:41:25.119 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-01-25T11:41:25.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.314+0000 7fb6c0407700 1 Processor -- start 2023-01-25T11:41:25.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.314+0000 7fb6c0407700 1 -- start start 2023-01-25T11:41:25.120 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6c0407700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:25.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6c0407700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b8077f30 con 0x7fb6b81026d0 2023-01-25T11:41:25.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6be1a3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:25.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6be1a3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8102ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50616/0 (socket says 172.21.15.119:50616) 2023-01-25T11:41:25.121 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6be1a3700 1 -- 172.21.15.119:0/3671984758 learned_addr learned my addr 172.21.15.119:0/3671984758 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:25.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6be1a3700 1 -- 172.21.15.119:0/3671984758 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b8103020 con 0x7fb6b81026d0 2023-01-25T11:41:25.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6be1a3700 1 --2- 172.21.15.119:0/3671984758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8102ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40098a0 tx=0x7fb6b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0a4890bc1b8e68 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:25.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6bd1a1700 1 -- 172.21.15.119:0/3671984758 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4004160 con 0x7fb6b81026d0 2023-01-25T11:41:25.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6bd1a1700 1 -- 172.21.15.119:0/3671984758 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fb6b40042c0 con 0x7fb6b81026d0 2023-01-25T11:41:25.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.315+0000 7fb6bd1a1700 1 -- 172.21.15.119:0/3671984758 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6b4011830 con 0x7fb6b81026d0 2023-01-25T11:41:25.122 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.316+0000 7fb6c0407700 1 -- 172.21.15.119:0/3671984758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 msgr2=0x7fb6b8102ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:25.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.316+0000 7fb6c0407700 1 --2- 172.21.15.119:0/3671984758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8102ae0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb6b40098a0 tx=0x7fb6b4009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:25.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.316+0000 7fb6c0407700 1 -- 172.21.15.119:0/3671984758 shutdown_connections 2023-01-25T11:41:25.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.316+0000 7fb6c0407700 1 --2- 172.21.15.119:0/3671984758 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8102ae0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:25.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.316+0000 7fb6c0407700 1 -- 172.21.15.119:0/3671984758 >> 172.21.15.119:0/3671984758 conn(0x7fb6b80fe220 msgr2=0x7fb6b8100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:25.123 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.316+0000 7fb6c0407700 1 -- 172.21.15.119:0/3671984758 shutdown_connections 2023-01-25T11:41:25.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.316+0000 7fb6c0407700 1 -- 172.21.15.119:0/3671984758 wait complete. 2023-01-25T11:41:25.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.317+0000 7fb6c0407700 1 Processor -- start 2023-01-25T11:41:25.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.317+0000 7fb6c0407700 1 -- start start 2023-01-25T11:41:25.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.317+0000 7fb6c0407700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:25.124 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.317+0000 7fb6c0407700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6b819a2b0 con 0x7fb6b81026d0 2023-01-25T11:41:25.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.317+0000 7fb6be1a3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:25.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.317+0000 7fb6be1a3700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50622/0 (socket says 172.21.15.119:50622) 2023-01-25T11:41:25.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.317+0000 7fb6be1a3700 1 -- 172.21.15.119:0/2403031069 learned_addr learned my addr 172.21.15.119:0/2403031069 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:25.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.318+0000 7fb6be1a3700 1 -- 172.21.15.119:0/2403031069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4009550 con 0x7fb6b81026d0 2023-01-25T11:41:25.125 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.318+0000 7fb6be1a3700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8199d70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009870 tx=0x7fb6b4011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:25.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.318+0000 7fb6ab7fe700 1 -- 172.21.15.119:0/2403031069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6b40036a0 con 0x7fb6b81026d0 2023-01-25T11:41:25.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.318+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6b8103fa0 con 0x7fb6b81026d0 2023-01-25T11:41:25.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.318+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6b8103e10 con 0x7fb6b81026d0 2023-01-25T11:41:25.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.318+0000 7fb6ab7fe700 1 -- 172.21.15.119:0/2403031069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fb6b4003800 con 0x7fb6b81026d0 2023-01-25T11:41:25.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.318+0000 7fb6ab7fe700 1 -- 172.21.15.119:0/2403031069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb6b40193c0 con 0x7fb6b81026d0 2023-01-25T11:41:25.126 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c005110 con 0x7fb6b81026d0 2023-01-25T11:41:25.127 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6ab7fe700 1 -- 172.21.15.119:0/2403031069 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 43522+0+0 (secure 0 0 0) 0x7fb6b4017020 con 0x7fb6b81026d0 2023-01-25T11:41:25.127 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6ab7fe700 20 mgrc handle_mgr_map mgrmap(e 12) v1 2023-01-25T11:41:25.127 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6ab7fe700 4 mgrc handle_mgr_map Got map version 12 2023-01-25T11:41:25.127 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6ab7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:25.127 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6ab7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:25.128 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6ab7fe700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fb6a4036770 0x7fb6a4038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:25.128 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6ab7fe700 1 -- 172.21.15.119:0/2403031069 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb6b4049ee0 con 0x7fb6b81026d0 2023-01-25T11:41:25.128 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.319+0000 7fb6bd9a2700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fb6a4036770 0x7fb6a4038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:25.128 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.320+0000 7fb6bd9a2700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fb6a4036770 0x7fb6a4038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac006fd0 tx=0x7fb6ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:25.128 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.324+0000 7fb6ab7fe700 1 -- 172.21.15.119:0/2403031069 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fb6b40226d0 con 0x7fb6b81026d0 2023-01-25T11:41:25.129 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.529+0000 7fb6c0407700 20 mgrc start_command cmd: [{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}] 2023-01-25T11:41:25.129 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.529+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 --> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb69c005d60 con 0x7fb6a4036770 2023-01-25T11:41:25.129 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.634+0000 7fb6ab7fe700 1 -- 172.21.15.119:0/2403031069 <== mgr.14152 v2:172.21.15.119:6800/4087238518 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb69c005d60 con 0x7fb6a4036770 2023-01-25T11:41:25.129 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.634+0000 7fb6ab7fe700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:25.129 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 10 mgrc shutdown 2023-01-25T11:41:25.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fb6a4036770 msgr2=0x7fb6a4038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:25.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fb6a4036770 0x7fb6a4038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac006fd0 tx=0x7fb6ac006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:25.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 msgr2=0x7fb6b8199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:25.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8199d70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009870 tx=0x7fb6b4011910 comp rx=0 tx=0).stop 2023-01-25T11:41:25.130 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 shutdown_connections 2023-01-25T11:41:25.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fb6a4036770 0x7fb6a4038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:25.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 --2- 172.21.15.119:0/2403031069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6b81026d0 0x7fb6b8199d70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:25.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 >> 172.21.15.119:0/2403031069 conn(0x7fb6b80fe220 msgr2=0x7fb6b818e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:25.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 shutdown_connections 2023-01-25T11:41:25.131 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:24.636+0000 7fb6c0407700 1 -- 172.21.15.119:0/2403031069 wait complete. 2023-01-25T11:41:25.131 INFO:teuthology.orchestra.run.smithi119.stdout:Creating initial admin user... 2023-01-25T11:41:26.389 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:26 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' 2023-01-25T11:41:26.389 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:26 smithi119 ceph-mon[102162]: from='mgr.14152 172.21.15.119:0/1307915394' entity='mgr.a' 2023-01-25T11:41:26.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$4j3lbNcT6/TwMxFI.4WRRObG18biLHh7222Kzm0bZ81td51VPWBdW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1674646885, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-01-25T11:41:26.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.510+0000 7fbb16418700 1 Processor -- start 2023-01-25T11:41:26.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.510+0000 7fbb16418700 1 -- start start 2023-01-25T11:41:26.465 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.510+0000 7fbb16418700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:26.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.510+0000 7fbb16418700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb10077f50 con 0x7fbb101048c0 2023-01-25T11:41:26.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.510+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:26.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.510+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10106ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50636/0 (socket says 172.21.15.119:50636) 2023-01-25T11:41:26.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.510+0000 7fbb0ffff700 1 -- 172.21.15.119:0/2737928806 learned_addr learned my addr 172.21.15.119:0/2737928806 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:26.466 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb0ffff700 1 -- 172.21.15.119:0/2737928806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb101071e0 con 0x7fbb101048c0 2023-01-25T11:41:26.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb0ffff700 1 --2- 172.21.15.119:0/2737928806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10106ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb000098a0 tx=0x7fbb00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f458b9380870fef6 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:26.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb0effd700 1 -- 172.21.15.119:0/2737928806 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb00004160 con 0x7fbb101048c0 2023-01-25T11:41:26.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb0effd700 1 -- 172.21.15.119:0/2737928806 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fbb000042c0 con 0x7fbb101048c0 2023-01-25T11:41:26.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb0effd700 1 -- 172.21.15.119:0/2737928806 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb00011830 con 0x7fbb101048c0 2023-01-25T11:41:26.467 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb16418700 1 -- 172.21.15.119:0/2737928806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 msgr2=0x7fbb10106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:26.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb16418700 1 --2- 172.21.15.119:0/2737928806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10106ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbb000098a0 tx=0x7fbb00009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:26.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb16418700 1 -- 172.21.15.119:0/2737928806 shutdown_connections 2023-01-25T11:41:26.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb16418700 1 --2- 172.21.15.119:0/2737928806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10106ca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:26.468 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb16418700 1 -- 172.21.15.119:0/2737928806 >> 172.21.15.119:0/2737928806 conn(0x7fbb100fe220 msgr2=0x7fbb10100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:26.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb16418700 1 -- 172.21.15.119:0/2737928806 shutdown_connections 2023-01-25T11:41:26.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.511+0000 7fbb16418700 1 -- 172.21.15.119:0/2737928806 wait complete. 2023-01-25T11:41:26.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.512+0000 7fbb16418700 1 Processor -- start 2023-01-25T11:41:26.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.512+0000 7fbb16418700 1 -- start start 2023-01-25T11:41:26.469 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.512+0000 7fbb16418700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:26.470 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.512+0000 7fbb16418700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb101980b0 con 0x7fbb101048c0 2023-01-25T11:41:26.470 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.512+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:26.470 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.512+0000 7fbb0ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50642/0 (socket says 172.21.15.119:50642) 2023-01-25T11:41:26.470 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.512+0000 7fbb0ffff700 1 -- 172.21.15.119:0/725584607 learned_addr learned my addr 172.21.15.119:0/725584607 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:26.470 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.513+0000 7fbb0ffff700 1 -- 172.21.15.119:0/725584607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb00009550 con 0x7fbb101048c0 2023-01-25T11:41:26.470 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.513+0000 7fbb0ffff700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10197b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb00003de0 tx=0x7fbb00011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:26.471 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.513+0000 7fbb0d7fa700 1 -- 172.21.15.119:0/725584607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb000037d0 con 0x7fbb101048c0 2023-01-25T11:41:26.471 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.513+0000 7fbb0d7fa700 1 -- 172.21.15.119:0/725584607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fbb00003930 con 0x7fbb101048c0 2023-01-25T11:41:26.471 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.513+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb10108150 con 0x7fbb101048c0 2023-01-25T11:41:26.471 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.513+0000 7fbb0d7fa700 1 -- 172.21.15.119:0/725584607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbb00024410 con 0x7fbb101048c0 2023-01-25T11:41:26.471 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.513+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb10107fc0 con 0x7fbb101048c0 2023-01-25T11:41:26.472 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0d7fa700 1 -- 172.21.15.119:0/725584607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 43522+0+0 (secure 0 0 0) 0x7fbb00017020 con 0x7fbb101048c0 2023-01-25T11:41:26.472 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0d7fa700 20 mgrc handle_mgr_map mgrmap(e 12) v1 2023-01-25T11:41:26.472 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0d7fa700 4 mgrc handle_mgr_map Got map version 12 2023-01-25T11:41:26.472 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0d7fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:26.472 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0d7fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:26.473 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0d7fa700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fbaf8036810 0x7fbaf8038cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:26.473 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb10198af0 con 0x7fbb101048c0 2023-01-25T11:41:26.473 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0d7fa700 1 -- 172.21.15.119:0/725584607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbb00049e60 con 0x7fbb101048c0 2023-01-25T11:41:26.473 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.514+0000 7fbb0f7fe700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fbaf8036810 0x7fbaf8038cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:26.473 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.515+0000 7fbb0f7fe700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fbaf8036810 0x7fbaf8038cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:26.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.519+0000 7fbb0d7fa700 1 -- 172.21.15.119:0/725584607 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fbb0002d3f0 con 0x7fbb101048c0 2023-01-25T11:41:26.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.743+0000 7fbb16418700 20 mgrc start_command cmd: [{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}] 2023-01-25T11:41:26.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.743+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 --> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fbb101082e0 con 0x7fbaf8036810 2023-01-25T11:41:26.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.967+0000 7fbb0d7fa700 1 -- 172.21.15.119:0/725584607 <== mgr.14152 v2:172.21.15.119:6800/4087238518 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fbb101082e0 con 0x7fbaf8036810 2023-01-25T11:41:26.474 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.967+0000 7fbb0d7fa700 20 mgrc handle_command_reply tid 0 r 0 2023-01-25T11:41:26.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.968+0000 7fbb16418700 10 mgrc shutdown 2023-01-25T11:41:26.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fbaf8036810 msgr2=0x7fbaf8038cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:26.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fbaf8036810 0x7fbaf8038cc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbb04006fd0 tx=0x7fbb04006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:26.475 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 msgr2=0x7fbb10197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:26.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10197b70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbb00003de0 tx=0x7fbb00011910 comp rx=0 tx=0).stop 2023-01-25T11:41:26.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 shutdown_connections 2023-01-25T11:41:26.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fbaf8036810 0x7fbaf8038cc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:26.476 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 --2- 172.21.15.119:0/725584607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb101048c0 0x7fbb10197b70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:26.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 >> 172.21.15.119:0/725584607 conn(0x7fbb100fe220 msgr2=0x7fbb100ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:26.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 shutdown_connections 2023-01-25T11:41:26.477 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:25.969+0000 7fbb16418700 1 -- 172.21.15.119:0/725584607 wait complete. 2023-01-25T11:41:26.477 INFO:teuthology.orchestra.run.smithi119.stdout:Fetching dashboard port number... 2023-01-25T11:41:27.389 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:27 smithi119 ceph-mon[102162]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-01-25T11:41:27.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stdout 8443 2023-01-25T11:41:27.609 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.958+0000 7fee794cf700 1 Processor -- start 2023-01-25T11:41:27.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.958+0000 7fee794cf700 1 -- start start 2023-01-25T11:41:27.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee794cf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee741047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:27.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee794cf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee74104da0 con 0x7fee741043c0 2023-01-25T11:41:27.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee741047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:27.610 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee741047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55532/0 (socket says 172.21.15.119:55532) 2023-01-25T11:41:27.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee72ffd700 1 -- 172.21.15.119:0/3705784824 learned_addr learned my addr 172.21.15.119:0/3705784824 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:27.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee72ffd700 1 -- 172.21.15.119:0/3705784824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee741028f0 con 0x7fee741043c0 2023-01-25T11:41:27.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee72ffd700 1 --2- 172.21.15.119:0/3705784824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee741047d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee640098a0 tx=0x7fee64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dac35345301f0d3 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:27.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee71ffb700 1 -- 172.21.15.119:0/3705784824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee64004160 con 0x7fee741043c0 2023-01-25T11:41:27.611 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee71ffb700 1 -- 172.21.15.119:0/3705784824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fee640042c0 con 0x7fee741043c0 2023-01-25T11:41:27.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.959+0000 7fee71ffb700 1 -- 172.21.15.119:0/3705784824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee64011830 con 0x7fee741043c0 2023-01-25T11:41:27.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.960+0000 7fee794cf700 1 -- 172.21.15.119:0/3705784824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 msgr2=0x7fee741047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:27.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.960+0000 7fee794cf700 1 --2- 172.21.15.119:0/3705784824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee741047d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fee640098a0 tx=0x7fee64009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:27.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.960+0000 7fee794cf700 1 -- 172.21.15.119:0/3705784824 shutdown_connections 2023-01-25T11:41:27.612 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.960+0000 7fee794cf700 1 --2- 172.21.15.119:0/3705784824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee741047d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:27.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.960+0000 7fee794cf700 1 -- 172.21.15.119:0/3705784824 >> 172.21.15.119:0/3705784824 conn(0x7fee740fe220 msgr2=0x7fee74100690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:27.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.960+0000 7fee794cf700 1 -- 172.21.15.119:0/3705784824 shutdown_connections 2023-01-25T11:41:27.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.960+0000 7fee794cf700 1 -- 172.21.15.119:0/3705784824 wait complete. 2023-01-25T11:41:27.613 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.961+0000 7fee794cf700 1 Processor -- start 2023-01-25T11:41:27.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.961+0000 7fee794cf700 1 -- start start 2023-01-25T11:41:27.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.961+0000 7fee794cf700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee74077b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:27.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.961+0000 7fee794cf700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee741986e0 con 0x7fee741043c0 2023-01-25T11:41:27.614 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.961+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee74077b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:27.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.961+0000 7fee72ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee74077b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55544/0 (socket says 172.21.15.119:55544) 2023-01-25T11:41:27.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.961+0000 7fee72ffd700 1 -- 172.21.15.119:0/1839793567 learned_addr learned my addr 172.21.15.119:0/1839793567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:27.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.962+0000 7fee72ffd700 1 -- 172.21.15.119:0/1839793567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64009550 con 0x7fee741043c0 2023-01-25T11:41:27.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.962+0000 7fee72ffd700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee74077b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee64009fb0 tx=0x7fee64011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:27.615 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.962+0000 7fee5bfff700 1 -- 172.21.15.119:0/1839793567 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee640036a0 con 0x7fee741043c0 2023-01-25T11:41:27.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.962+0000 7fee5bfff700 1 -- 172.21.15.119:0/1839793567 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7fee64003800 con 0x7fee741043c0 2023-01-25T11:41:27.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.962+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee74107100 con 0x7fee741043c0 2023-01-25T11:41:27.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.962+0000 7fee5bfff700 1 -- 172.21.15.119:0/1839793567 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fee64024410 con 0x7fee741043c0 2023-01-25T11:41:27.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.962+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee741036e0 con 0x7fee741043c0 2023-01-25T11:41:27.616 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee5bfff700 1 -- 172.21.15.119:0/1839793567 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 43522+0+0 (secure 0 0 0) 0x7fee64017020 con 0x7fee741043c0 2023-01-25T11:41:27.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee5bfff700 20 mgrc handle_mgr_map mgrmap(e 12) v1 2023-01-25T11:41:27.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee74107290 con 0x7fee741043c0 2023-01-25T11:41:27.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee5bfff700 4 mgrc handle_mgr_map Got map version 12 2023-01-25T11:41:27.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee5bfff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:27.617 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee5bfff700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:27.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee5bfff700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fee5c0367c0 0x7fee5c038c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:27.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee5bfff700 1 -- 172.21.15.119:0/1839793567 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fee6404a9f0 con 0x7fee741043c0 2023-01-25T11:41:27.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee727fc700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fee5c0367c0 0x7fee5c038c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:27.618 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.963+0000 7fee727fc700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fee5c0367c0 0x7fee5c038c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee68006fd0 tx=0x7fee68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:27.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:26.968+0000 7fee5bfff700 1 -- 172.21.15.119:0/1839793567 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7fee64018940 con 0x7fee741043c0 2023-01-25T11:41:27.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.170+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fee74061b00 con 0x7fee741043c0 2023-01-25T11:41:27.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.171+0000 7fee5bfff700 1 -- 172.21.15.119:0/1839793567 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fee64018350 con 0x7fee741043c0 2023-01-25T11:41:27.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.172+0000 7fee794cf700 10 mgrc shutdown 2023-01-25T11:41:27.619 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.172+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fee5c0367c0 msgr2=0x7fee5c038c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:27.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.172+0000 7fee794cf700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fee5c0367c0 0x7fee5c038c70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee68006fd0 tx=0x7fee68006e40 comp rx=0 tx=0).stop 2023-01-25T11:41:27.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.172+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 msgr2=0x7fee74077b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:27.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.172+0000 7fee794cf700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee74077b50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee64009fb0 tx=0x7fee64011910 comp rx=0 tx=0).stop 2023-01-25T11:41:27.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.173+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 shutdown_connections 2023-01-25T11:41:27.620 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.173+0000 7fee794cf700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7fee5c0367c0 0x7fee5c038c70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:27.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.173+0000 7fee794cf700 1 --2- 172.21.15.119:0/1839793567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee741043c0 0x7fee74077b50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:27.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.173+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 >> 172.21.15.119:0/1839793567 conn(0x7fee740fe220 msgr2=0x7fee740ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:27.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.173+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 shutdown_connections 2023-01-25T11:41:27.621 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:27.173+0000 7fee794cf700 1 -- 172.21.15.119:0/1839793567 wait complete. 2023-01-25T11:41:27.622 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-01-25T11:41:27.622 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout disabled 2023-01-25T11:41:27.624 INFO:teuthology.orchestra.run.smithi119.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-01-25T11:41:27.624 INFO:teuthology.orchestra.run.smithi119.stdout:systemctl: stdout inactive 2023-01-25T11:41:27.625 INFO:teuthology.orchestra.run.smithi119.stdout:firewalld.service is not enabled 2023-01-25T11:41:27.625 INFO:teuthology.orchestra.run.smithi119.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-01-25T11:41:27.625 INFO:teuthology.orchestra.run.smithi119.stdout:Ceph Dashboard is now available at: 2023-01-25T11:41:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:41:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: URL: https://smithi119.front.sepia.ceph.com:8443/ 2023-01-25T11:41:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: User: admin 2023-01-25T11:41:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: Password: pculkj8xmv 2023-01-25T11:41:27.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-01-25T11:41:27.627 INFO:teuthology.orchestra.run.smithi119.stdout:Saving cluster configuration to /var/lib/ceph/f64523de-9ca4-11ed-9e56-001a4aab830c/config directory 2023-01-25T11:41:27.627 INFO:teuthology.orchestra.run.smithi119.stdout:Enabling autotune for osd_memory_target 2023-01-25T11:41:28.390 INFO:journalctl@ceph.mon.a.smithi119.stdout:Jan 25 11:41:28 smithi119 ceph-mon[102162]: from='client.? 172.21.15.119:0/1839793567' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-01-25T11:41:28.667 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.014+0000 7f0de763a700 1 Processor -- start 2023-01-25T11:41:28.667 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.014+0000 7f0de763a700 1 -- start start 2023-01-25T11:41:28.667 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.014+0000 7f0de763a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de01042a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:28.668 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.014+0000 7f0de763a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de01047e0 con 0x7f0de0103e90 2023-01-25T11:41:28.668 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.014+0000 7f0de53d6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de01042a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:28.668 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.014+0000 7f0de53d6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de01042a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55554/0 (socket says 172.21.15.119:55554) 2023-01-25T11:41:28.668 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.014+0000 7f0de53d6700 1 -- 172.21.15.119:0/1836325646 learned_addr learned my addr 172.21.15.119:0/1836325646 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:28.668 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de53d6700 1 -- 172.21.15.119:0/1836325646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de0104920 con 0x7f0de0103e90 2023-01-25T11:41:28.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de53d6700 1 --2- 172.21.15.119:0/1836325646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de01042a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098a0 tx=0x7f0ddc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a4e4396077052 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:28.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0dd7fff700 1 -- 172.21.15.119:0/1836325646 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ddc004160 con 0x7f0de0103e90 2023-01-25T11:41:28.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0dd7fff700 1 -- 172.21.15.119:0/1836325646 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f0ddc0042c0 con 0x7f0de0103e90 2023-01-25T11:41:28.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0dd7fff700 1 -- 172.21.15.119:0/1836325646 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ddc011830 con 0x7f0de0103e90 2023-01-25T11:41:28.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de763a700 1 -- 172.21.15.119:0/1836325646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 msgr2=0x7f0de01042a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:28.669 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de763a700 1 --2- 172.21.15.119:0/1836325646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de01042a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc0098a0 tx=0x7f0ddc009bb0 comp rx=0 tx=0).stop 2023-01-25T11:41:28.670 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de763a700 1 -- 172.21.15.119:0/1836325646 shutdown_connections 2023-01-25T11:41:28.670 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de763a700 1 --2- 172.21.15.119:0/1836325646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de01042a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-01-25T11:41:28.670 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de763a700 1 -- 172.21.15.119:0/1836325646 >> 172.21.15.119:0/1836325646 conn(0x7f0de00ff410 msgr2=0x7f0de0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-01-25T11:41:28.670 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.015+0000 7f0de763a700 1 -- 172.21.15.119:0/1836325646 shutdown_connections 2023-01-25T11:41:28.670 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de763a700 1 -- 172.21.15.119:0/1836325646 wait complete. 2023-01-25T11:41:28.671 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de763a700 1 Processor -- start 2023-01-25T11:41:28.671 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de763a700 1 -- start start 2023-01-25T11:41:28.671 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de763a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de0197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:28.671 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de763a700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de0197fb0 con 0x7f0de0103e90 2023-01-25T11:41:28.671 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de53d6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de0197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:28.672 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de53d6700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de0197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55566/0 (socket says 172.21.15.119:55566) 2023-01-25T11:41:28.672 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.016+0000 7f0de53d6700 1 -- 172.21.15.119:0/2505681949 learned_addr learned my addr 172.21.15.119:0/2505681949 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-01-25T11:41:28.672 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.017+0000 7f0de53d6700 1 -- 172.21.15.119:0/2505681949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc009550 con 0x7f0de0103e90 2023-01-25T11:41:28.672 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.017+0000 7f0de53d6700 1 --2- 172.21.15.119:0/2505681949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0de0103e90 0x7f0de0197a70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009870 tx=0x7f0ddc011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:28.672 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.017+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ddc0036a0 con 0x7f0de0103e90 2023-01-25T11:41:28.673 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.017+0000 7f0de763a700 1 -- 172.21.15.119:0/2505681949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de0105dc0 con 0x7f0de0103e90 2023-01-25T11:41:28.673 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.017+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(24 keys) v1 ==== 1035+0+0 (secure 0 0 0) 0x7f0ddc003800 con 0x7f0de0103e90 2023-01-25T11:41:28.673 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.017+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0ddc0193c0 con 0x7f0de0103e90 2023-01-25T11:41:28.673 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.017+0000 7f0de763a700 1 -- 172.21.15.119:0/2505681949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de0105c30 con 0x7f0de0103e90 2023-01-25T11:41:28.673 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 12) v1 ==== 43522+0+0 (secure 0 0 0) 0x7f0ddc017020 con 0x7f0de0103e90 2023-01-25T11:41:28.674 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0dd67fc700 20 mgrc handle_mgr_map mgrmap(e 12) v1 2023-01-25T11:41:28.674 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0dd67fc700 4 mgrc handle_mgr_map Got map version 12 2023-01-25T11:41:28.674 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0dd67fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:28.674 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0dd67fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:28.674 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0dd67fc700 1 --2- 172.21.15.119:0/2505681949 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7f0dcc036770 0x7f0dcc038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-01-25T11:41:28.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0de763a700 1 -- 172.21.15.119:0/2505681949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de0108210 con 0x7f0de0103e90 2023-01-25T11:41:28.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0ddc049e30 con 0x7f0de0103e90 2023-01-25T11:41:28.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0de4bd5700 1 --2- 172.21.15.119:0/2505681949 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7f0dcc036770 0x7f0dcc038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-01-25T11:41:28.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.018+0000 7f0de4bd5700 1 --2- 172.21.15.119:0/2505681949 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7f0dcc036770 0x7f0dcc038c20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0dd0006fd0 tx=0x7f0dd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-01-25T11:41:28.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.023+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173756 (secure 0 0 0) 0x7f0ddc019650 con 0x7f0de0103e90 2023-01-25T11:41:28.675 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.136+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 13) v1 ==== 43566+0+0 (secure 0 0 0) 0x7f0ddc00f050 con 0x7f0de0103e90 2023-01-25T11:41:28.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.136+0000 7f0dd67fc700 20 mgrc handle_mgr_map mgrmap(e 13) v1 2023-01-25T11:41:28.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.136+0000 7f0dd67fc700 4 mgrc handle_mgr_map Got map version 13 2023-01-25T11:41:28.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.136+0000 7f0dd67fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] 2023-01-25T11:41:28.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.210+0000 7f0de763a700 1 -- 172.21.15.119:0/2505681949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f0de0061b00 con 0x7f0de0103e90 2023-01-25T11:41:28.676 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.211+0000 7f0dd67fc700 1 -- 172.21.15.119:0/2505681949 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0ddc0320c0 con 0x7f0de0103e90 2023-01-25T11:41:28.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.212+0000 7f0de763a700 10 mgrc shutdown 2023-01-25T11:41:28.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.212+0000 7f0de763a700 1 -- 172.21.15.119:0/2505681949 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7f0dcc036770 msgr2=0x7f0dcc038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-01-25T11:41:28.677 INFO:teuthology.orchestra.run.smithi119.stdout:/usr/bin/ceph: stderr 2023-01-25T11:41:28.212+0000 7f0de763a700 1 --2- 172.21.15.119:0/2505681949 >> [v2:172.21.15.119:6800/4087238518,v1:172.21.15.119:6801/4087238518] conn(0x7f0dcc