2022-11-27T15:53:18.876 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T15:53:18.878 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088633 --description fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} --owner scheduled_yuriw@teuthology --verbose --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088633/orig.config.yaml 2022-11-27T15:53:18.931 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T15:53:19.008 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088633 branch: main description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088633' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: main ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi105.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/l60JzpJHr1FbtS4dRcaBYv0m0w0J/Ltydp031jdNpwQxyOKVdA1Fyhzb6555nk4Tm7iRp9Ib74f/Xf96yzaTppE7ERQt2x3I9UpdIAl5v3FS5S0/4Rznn7rYRMincqoX0mBvZoEpS0niViHHCwqpWDm5ahZlv+N3fQe+qPLfAtBxH0pjQTjY3DqZtbgLW332FOc2hohJxMSLq8I4G1ROLzyE8q+SGfIUoFbIPqqTZzU9CqcUhUNYr4Cbw4l63PKrPoHPhnTe9es3fLQ3UbvUiBMBkgnqSqmH2Hx4CoGgLNtJGdUdaeBEL16K94XIyJGNHUhG3CNZIqxFC5qySENQPCtM2SrJpD2vfaY35SHtpM3pd6BopXmmmob+kG9//aIr7fdgyj2LabB2CbXjxe/vWlNhp/jmGpQfJLV1Fjpp41FlKNFXfJBKxMLBMyaT7RdLwJPHX9qKIyl5SBTsq3rJgd9x6AtWk+CEYdITJbIB/scgfjWt809zTqGkZExsBcc= smithi170.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrA3uoaZw6MOkND0hyZkDHakwB9ke54CwrLdee1vAdOHd6YDdkFnnqQFHn6ohqeIE2mIOONsv8CItMKdOAKxS+oZeAwXxFt6JlPQ1VR/NUUNIvgvJt5XoNaEIZ5vSOHuif8tvKueMvX8pAIgLPyc7gQHrv9CfFSrLVjsDsHjqYqHSXvDKPxrUwky2NnNJ6eAiAVZQeR+JRcN7IaD+Z5SOlP/dKl/PQez5f4f6xI5djYIM2uxVaQLrF/DHPWgzSScTu0GlWtkwhUo6AOTn2VR9nfpz85q1HD6AWy4951ZXkOQ37DM4pY2Ja652X6+nK4THfhA5MZCzpZqBW2rUC0ZECzfixVcIKLC0KROLt45z5DQ2vgNChb5JJsXd2c/8Ls9FGXu6hnYnVAitW+z+x79Y9wdz8zZMaqI4gqHL+VYk7QOftb8ATjRgbv5MAn7d6RP0r0pw4fYN6bJV+y4mdLUy4DuGwHHByWqK0TgBT2ppIPAJjZaOLxTdFoww3BmEnY80= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.20399 2022-11-27T15:53:19.009 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T15:53:19.010 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T15:53:19.010 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T15:53:19.016 INFO:teuthology.task.internal:Checking packages... 2022-11-27T15:53:19.041 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T15:53:19.041 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T15:53:19.042 INFO:teuthology.packaging:ref: None 2022-11-27T15:53:19.042 INFO:teuthology.packaging:tag: None 2022-11-27T15:53:19.043 INFO:teuthology.packaging:branch: main 2022-11-27T15:53:19.043 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T15:53:19.043 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T15:53:19.268 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-982.g2bb1aa26 2022-11-27T15:53:19.268 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T15:53:19.275 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T15:53:19.276 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T15:53:19.322 INFO:teuthology.task.internal:Saving configuration 2022-11-27T15:53:19.341 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T15:53:19.352 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T15:53:19.379 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088633', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 15:48:27.495809', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBACcy5239YjZmAOvzBgzb2DB8DYzPutMdEzOdnfvfpMEsCDgWY5KZB5eF1V2QLvvtNbdNNB6CJX2iY580r6XVD4='} 2022-11-27T15:53:19.406 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088633', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 15:48:27.494229', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHgrg6yh/QtuBmNIxNX5m6zoM7Ivu19H75SZDKtqjUjiDTSQNV9kGU/nNbQFZYdOG0pHycVV33rtR9Sx/WQ9ZbU='} 2022-11-27T15:53:19.407 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T15:53:19.414 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-11-27T15:53:19.414 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-11-27T15:53:19.415 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T15:53:19.513 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f007baec840>, signals=[15]) 2022-11-27T15:53:19.515 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T15:53:19.522 INFO:teuthology.task.internal:Opening connections... 2022-11-27T15:53:19.522 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2022-11-27T15:53:19.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T15:53:19.611 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2022-11-27T15:53:19.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T15:53:19.706 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T15:53:19.714 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2022-11-27T15:53:19.736 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2022-11-27T15:53:19.736 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2022-11-27T15:53:19.796 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2022-11-27T15:53:19.796 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="8" 2022-11-27T15:53:19.796 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2022-11-27T15:53:19.797 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2022-11-27T15:53:19.797 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="8" 2022-11-27T15:53:19.798 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el8" 2022-11-27T15:53:19.798 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T15:53:19.799 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2022-11-27T15:53:19.799 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T15:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2022-11-27T15:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T15:53:19.800 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T15:53:19.801 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T15:53:19.802 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2022-11-27T15:53:19.824 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2022-11-27T15:53:19.845 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2022-11-27T15:53:19.845 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2022-11-27T15:53:19.904 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2022-11-27T15:53:19.904 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="8" 2022-11-27T15:53:19.905 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2022-11-27T15:53:19.905 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2022-11-27T15:53:19.905 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="8" 2022-11-27T15:53:19.905 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el8" 2022-11-27T15:53:19.906 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 8" 2022-11-27T15:53:19.906 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2022-11-27T15:53:19.906 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2022-11-27T15:53:19.906 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2022-11-27T15:53:19.907 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T15:53:19.907 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T15:53:19.907 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2022-11-27T15:53:19.908 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2022-11-27T15:53:19.930 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T15:53:19.942 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T15:53:19.965 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T15:53:19.966 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2022-11-27T15:53:19.970 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2022-11-27T15:53:19.990 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T15:53:19.997 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T15:53:19.997 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2022-11-27T15:53:20.029 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2022-11-27T15:53:20.058 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T15:53:20.115 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T15:53:20.133 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-11-27T15:53:20.133 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-11-27T15:53:20.134 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T15:53:20.135 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T15:53:20.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:53:20.154 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2022-11-27T15:53:20.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:53:20.155 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2022-11-27T15:53:20.213 INFO:teuthology.orchestra.run.smithi170.stdout:4.18.0-408.el8.x86_64 2022-11-27T15:53:20.214 INFO:teuthology.task.kernel:Running kernel on smithi170: 4.18.0-408.el8.x86_64 2022-11-27T15:53:20.214 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2022-11-27T15:53:20.216 INFO:teuthology.orchestra.run.smithi105.stdout:4.18.0-408.el8.x86_64 2022-11-27T15:53:20.216 INFO:teuthology.task.kernel:Running kernel on smithi105: 4.18.0-408.el8.x86_64 2022-11-27T15:53:20.216 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum install -y kernel 2022-11-27T15:53:23.342 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - AppStream 35 MB/s | 26 MB 00:00 2022-11-27T15:53:23.487 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - AppStream 30 MB/s | 26 MB 00:00 2022-11-27T15:53:28.709 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - BaseOS 36 MB/s | 26 MB 00:00 2022-11-27T15:53:28.837 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - BaseOS 36 MB/s | 26 MB 00:00 2022-11-27T15:53:32.733 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - Extras 176 kB/s | 18 kB 00:00 2022-11-27T15:53:32.811 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - Extras 164 kB/s | 18 kB 00:00 2022-11-27T15:53:33.085 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - PowerTools 22 MB/s | 5.3 MB 00:00 2022-11-27T15:53:33.172 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - PowerTools 22 MB/s | 5.3 MB 00:00 2022-11-27T15:53:36.596 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T15:53:36.626 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T15:53:37.771 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 12 MB/s | 13 MB 00:01 2022-11-27T15:53:37.834 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 12 MB/s | 13 MB 00:01 2022-11-27T15:53:40.972 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 299 kB/s | 24 kB 00:00 2022-11-27T15:53:41.026 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 591 kB/s | 24 kB 00:00 2022-11-27T15:53:43.899 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T15:53:43.900 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T15:53:43.919 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-4.18.0-408.el8.x86_64 is already installed. 2022-11-27T15:53:43.920 INFO:teuthology.orchestra.run.smithi105.stdout:Package kernel-4.18.0-305.3.1.el8.x86_64 is already installed. 2022-11-27T15:53:44.035 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2022-11-27T15:53:44.036 INFO:teuthology.orchestra.run.smithi170.stdout:Nothing to do. 2022-11-27T15:53:44.036 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2022-11-27T15:53:44.061 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2022-11-27T15:53:44.062 INFO:teuthology.orchestra.run.smithi105.stdout:Nothing to do. 2022-11-27T15:53:44.062 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2022-11-27T15:53:44.139 DEBUG:teuthology.orchestra.run.smithi170:> echo no | sudo yum reinstall kernel || true 2022-11-27T15:53:44.200 DEBUG:teuthology.orchestra.run.smithi105:> echo no | sudo yum reinstall kernel || true 2022-11-27T15:53:44.722 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 03:53:40 PM UTC. 2022-11-27T15:53:44.781 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:04 ago on Sun 27 Nov 2022 03:53:40 PM UTC. 2022-11-27T15:53:45.803 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2022-11-27T15:53:45.804 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2022-11-27T15:53:45.804 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2022-11-27T15:53:45.804 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2022-11-27T15:53:45.805 INFO:teuthology.orchestra.run.smithi170.stdout:Reinstalling: 2022-11-27T15:53:45.805 INFO:teuthology.orchestra.run.smithi170.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T15:53:45.805 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T15:53:45.805 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2022-11-27T15:53:45.806 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2022-11-27T15:53:45.806 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T15:53:45.806 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 8.5 M 2022-11-27T15:53:45.806 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 0 2022-11-27T15:53:45.807 INFO:teuthology.orchestra.run.smithi170.stderr:Operation aborted. 2022-11-27T15:53:45.861 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2022-11-27T15:53:45.863 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2022-11-27T15:53:45.863 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2022-11-27T15:53:45.863 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2022-11-27T15:53:45.863 INFO:teuthology.orchestra.run.smithi105.stdout:Reinstalling: 2022-11-27T15:53:45.864 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T15:53:45.864 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T15:53:45.864 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2022-11-27T15:53:45.864 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2022-11-27T15:53:45.865 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T15:53:45.865 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 8.5 M 2022-11-27T15:53:45.865 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 0 2022-11-27T15:53:45.865 INFO:teuthology.orchestra.run.smithi105.stderr:Operation aborted. 2022-11-27T15:53:45.874 INFO:teuthology.orchestra.run.smithi170.stdout:Is this ok [y/N]: 2022-11-27T15:53:45.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum reinstall -y kernel || true 2022-11-27T15:53:45.928 INFO:teuthology.orchestra.run.smithi105.stdout:Is this ok [y/N]: 2022-11-27T15:53:45.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum reinstall -y kernel || true 2022-11-27T15:53:46.457 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:06 ago on Sun 27 Nov 2022 03:53:40 PM UTC. 2022-11-27T15:53:46.503 INFO:teuthology.orchestra.run.smithi105.stdout:Last metadata expiration check: 0:00:06 ago on Sun 27 Nov 2022 03:53:40 PM UTC. 2022-11-27T15:53:47.530 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2022-11-27T15:53:47.532 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2022-11-27T15:53:47.532 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2022-11-27T15:53:47.532 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2022-11-27T15:53:47.532 INFO:teuthology.orchestra.run.smithi170.stdout:Reinstalling: 2022-11-27T15:53:47.533 INFO:teuthology.orchestra.run.smithi170.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T15:53:47.533 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T15:53:47.533 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2022-11-27T15:53:47.533 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2022-11-27T15:53:47.534 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T15:53:47.534 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 8.5 M 2022-11-27T15:53:47.534 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 0 2022-11-27T15:53:47.534 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2022-11-27T15:53:47.578 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2022-11-27T15:53:47.579 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2022-11-27T15:53:47.579 INFO:teuthology.orchestra.run.smithi105.stdout: Package Architecture Version Repository Size 2022-11-27T15:53:47.580 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2022-11-27T15:53:47.580 INFO:teuthology.orchestra.run.smithi105.stdout:Reinstalling: 2022-11-27T15:53:47.581 INFO:teuthology.orchestra.run.smithi105.stdout: kernel x86_64 4.18.0-408.el8 CentOS-BaseOS 8.5 M 2022-11-27T15:53:47.581 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T15:53:47.582 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2022-11-27T15:53:47.582 INFO:teuthology.orchestra.run.smithi105.stdout:================================================================================ 2022-11-27T15:53:47.582 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T15:53:47.583 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 8.5 M 2022-11-27T15:53:47.583 INFO:teuthology.orchestra.run.smithi105.stdout:Installed size: 0 2022-11-27T15:53:47.584 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2022-11-27T15:53:47.744 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-4.18.0-408.el8.x86_64.rpm 40 MB/s | 8.5 MB 00:00 2022-11-27T15:53:47.745 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:53:47.746 INFO:teuthology.orchestra.run.smithi170.stdout:Total 40 MB/s | 8.5 MB 00:00 2022-11-27T15:53:47.840 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2022-11-27T15:53:47.844 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-4.18.0-408.el8.x86_64.rpm 32 MB/s | 8.5 MB 00:00 2022-11-27T15:53:47.845 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2022-11-27T15:53:47.845 INFO:teuthology.orchestra.run.smithi105.stdout:Total 32 MB/s | 8.5 MB 00:00 2022-11-27T15:53:47.945 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2022-11-27T15:53:48.254 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2022-11-27T15:53:48.254 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2022-11-27T15:53:48.372 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2022-11-27T15:53:48.372 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2022-11-27T15:53:48.425 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2022-11-27T15:53:48.436 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2022-11-27T15:53:48.551 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2022-11-27T15:53:48.566 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2022-11-27T15:53:48.710 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2022-11-27T15:53:48.845 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2022-11-27T15:53:48.925 INFO:teuthology.orchestra.run.smithi170.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T15:53:48.926 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T15:53:49.076 INFO:teuthology.orchestra.run.smithi105.stdout: Reinstalling : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T15:53:49.076 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T15:53:49.304 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T15:53:49.305 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T15:53:49.478 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T15:53:49.478 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 1/2 2022-11-27T15:53:49.791 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T15:53:49.792 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T15:53:49.792 INFO:teuthology.orchestra.run.smithi170.stdout:Reinstalled: 2022-11-27T15:53:49.793 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T15:53:49.793 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T15:53:49.793 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2022-11-27T15:53:49.890 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel --last | head -n 1 2022-11-27T15:53:49.992 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : kernel-4.18.0-408.el8.x86_64 2/2 2022-11-27T15:53:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T15:53:49.993 INFO:teuthology.orchestra.run.smithi105.stdout:Reinstalled: 2022-11-27T15:53:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: kernel-4.18.0-408.el8.x86_64 2022-11-27T15:53:49.993 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T15:53:49.994 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2022-11-27T15:53:50.024 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 03:53:48 PM UTC 2022-11-27T15:53:50.026 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T15:53:50.027 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T15:53:50.027 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-11-27T15:53:50.028 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T15:53:50.080 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2022-11-27T15:53:50.096 DEBUG:teuthology.parallel:result is None 2022-11-27T15:53:50.104 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q kernel --last | head -n 1 2022-11-27T15:53:50.233 INFO:teuthology.orchestra.run.smithi105.stdout:kernel-4.18.0-408.el8.x86_64 Sun 27 Nov 2022 03:53:48 PM UTC 2022-11-27T15:53:50.235 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-408.el8.x86_64 2022-11-27T15:53:50.235 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T15:53:50.235 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-11-27T15:53:50.236 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T15:53:50.268 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2022-11-27T15:53:50.281 DEBUG:teuthology.parallel:result is None 2022-11-27T15:53:50.281 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T15:53:50.293 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T15:53:50.294 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T15:53:50.324 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T15:53:50.350 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T15:53:50.372 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T15:53:50.407 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T15:53:50.408 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T15:53:50.414 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T15:53:50.453 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T15:53:50.462 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T15:53:50.462 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T15:53:50.478 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T15:53:50.521 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:53:50.530 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:53:50.531 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:53:50.539 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T15:53:50.541 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T15:53:50.549 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T15:53:50.549 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T15:53:50.552 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T15:53:50.601 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T15:53:50.621 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T15:53:50.622 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T15:53:50.625 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T15:53:50.660 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:53:50.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:53:50.802 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:53:50.852 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:53:50.889 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T15:53:50.890 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T15:53:50.956 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:53:51.001 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T15:53:51.064 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:53:51.110 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T15:53:51.150 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T15:53:51.151 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T15:53:51.218 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2022-11-27T15:53:51.220 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2022-11-27T15:53:51.276 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T15:53:51.323 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T15:53:51.751 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T15:53:51.775 INFO:teuthology.task.internal:Starting timer... 2022-11-27T15:53:51.776 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T15:53:51.796 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T15:53:51.846 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T15:53:51.846 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2022-11-27T15:53:51.955 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2022-11-27T15:53:51.956 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2022-11-27T15:53:52.027 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2022-11-27T15:53:52.029 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T15:53:52.029 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2022-11-27T15:53:52.056 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2022-11-27T15:53:52.057 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2022-11-27T15:53:52.084 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2022-11-27T15:53:52.085 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi105.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2022-11-27T15:53:52.085 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T15:53:52.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:53:52.127 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T15:53:52.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T15:53:52.156 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T15:53:52.157 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2022-11-27T15:53:52.200 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2022-11-27T15:53:52.231 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T15:53:52.243 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2022-11-27T15:53:52.250 INFO:teuthology.repo_utils:Fetching main from origin 2022-11-27T15:53:52.368 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T15:53:52.401 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T15:53:52.402 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T16:00:53.745 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2022-11-27T16:00:53.782 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2022-11-27T16:00:53.785 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:00:53.868 DEBUG:teuthology.orchestra.run.smithi105:> true 2022-11-27T16:00:53.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2022-11-27T16:00:53.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:00:54.021 DEBUG:teuthology.orchestra.run.smithi170:> true 2022-11-27T16:00:54.090 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T16:00:54.103 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T16:00:54.104 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:00:54.107 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:00:54.109 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:00:54.110 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:00:54.138 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:00:54.152 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:00:54.163 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:00:54.177 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:00:54.217 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2022-11-27T16:00:54.227 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2022-11-27T16:00:54.242 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:00:54.245 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2022-11-27T16:00:54.255 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:00:54.256 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2022-11-27T16:00:54.271 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:00:54.284 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:00:54.315 INFO:teuthology.orchestra.run.smithi105.stderr:bash: ntpq: command not found 2022-11-27T16:00:54.319 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:00:54.320 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2022-11-27T16:00:54.320 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.320 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.321 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.321 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.349 INFO:teuthology.orchestra.run.smithi170.stderr:bash: ntpq: command not found 2022-11-27T16:00:54.354 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:00:54.354 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2022-11-27T16:00:54.355 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.355 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.355 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.355 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:00:54.356 INFO:teuthology.run_tasks:Running task install... 2022-11-27T16:00:54.371 DEBUG:teuthology.task.install:project ceph 2022-11-27T16:00:54.372 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T16:00:54.372 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T16:00:54.373 INFO:teuthology.task.install:Using flavor: default 2022-11-27T16:00:54.380 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T16:00:54.381 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-11-27T16:00:54.381 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2022-11-27T16:00:54.418 INFO:teuthology.orchestra.run.smithi105.stdout:CentOSStream 2022-11-27T16:00:54.419 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:00:54.420 DEBUG:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2022-11-27T16:00:54.421 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:00:54.421 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2022-11-27T16:00:54.458 INFO:teuthology.orchestra.run.smithi170.stdout:CentOSStream 2022-11-27T16:00:54.459 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:00:54.460 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:00:54.460 DEBUG:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2022-11-27T16:00:54.495 INFO:teuthology.orchestra.run.smithi105.stdout:CentOSStream 2022-11-27T16:00:54.497 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:00:54.498 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:00:54.547 INFO:teuthology.orchestra.run.smithi170.stdout:CentOSStream 2022-11-27T16:00:54.548 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:00:54.549 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:00:54.688 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:00:54.688 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:00:54.719 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:00:54.720 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:00:54.941 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:00:54.941 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:00:54.942 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:00:54.969 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:00:54.969 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:00:54.972 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:00:54.973 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:00:54.973 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:00:55.000 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:00:55.001 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:00:55.036 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:00:55.074 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:00:55.118 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:00:55.156 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:00:55.185 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2022-11-27T16:00:55.187 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2022-11-27T16:00:55.222 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2022-11-27T16:00:55.224 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2022-11-27T16:00:55.444 INFO:teuthology.orchestra.run.smithi105.stdout:64 files removed 2022-11-27T16:00:55.473 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:00:55.481 INFO:teuthology.orchestra.run.smithi170.stdout:64 files removed 2022-11-27T16:00:55.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:00:56.479 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 125 kB/s | 81 kB 00:00 2022-11-27T16:00:56.531 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 123 kB/s | 81 kB 00:00 2022-11-27T16:00:56.857 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 32 kB/s | 9.4 kB 00:00 2022-11-27T16:00:56.863 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 27 kB/s | 9.4 kB 00:00 2022-11-27T16:00:57.116 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 6.5 kB/s | 1.5 kB 00:00 2022-11-27T16:00:57.147 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 5.9 kB/s | 1.5 kB 00:00 2022-11-27T16:00:57.969 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - AppStream 31 MB/s | 26 MB 00:00 2022-11-27T16:00:57.992 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - AppStream 32 MB/s | 26 MB 00:00 2022-11-27T16:01:03.294 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - BaseOS 37 MB/s | 26 MB 00:00 2022-11-27T16:01:03.334 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - BaseOS 37 MB/s | 26 MB 00:00 2022-11-27T16:01:07.251 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - Extras 154 kB/s | 18 kB 00:00 2022-11-27T16:01:07.315 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - Extras 174 kB/s | 18 kB 00:00 2022-11-27T16:01:07.620 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T16:01:07.685 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-8 - PowerTools 21 MB/s | 5.3 MB 00:00 2022-11-27T16:01:12.877 INFO:teuthology.orchestra.run.smithi170.stdout:Copr repo for python3-asyncssh owned by ceph 833 B/s | 3.5 kB 00:04 2022-11-27T16:01:14.817 INFO:teuthology.orchestra.run.smithi105.stdout:Copr repo for python3-asyncssh owned by ceph 578 B/s | 3.5 kB 00:06 2022-11-27T16:01:16.584 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 13 MB 00:01 2022-11-27T16:01:18.677 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 2.3 MB/s | 13 MB 00:05 2022-11-27T16:01:19.809 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 240 kB/s | 24 kB 00:00 2022-11-27T16:01:21.818 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 513 kB/s | 24 kB 00:00 2022-11-27T16:01:22.919 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:01:22.920 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:01:22.931 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.40.0-1.el8.x86_64 is already installed. 2022-11-27T16:01:22.931 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-initscripts-updown-1:1.40.0-1.el8.noarch is already installed. 2022-11-27T16:01:22.932 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2022-11-27T16:01:22.933 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2022-11-27T16:01:22.933 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:01:22.935 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:01:22.937 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:01:22.938 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:01:23.054 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2022-11-27T16:01:23.061 INFO:teuthology.orchestra.run.smithi105.stdout:========================================================================================================= 2022-11-27T16:01:23.061 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2022-11-27T16:01:23.062 INFO:teuthology.orchestra.run.smithi105.stdout:========================================================================================================= 2022-11-27T16:01:23.062 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2022-11-27T16:01:23.062 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2022-11-27T16:01:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:01:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:01:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:01:23.063 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:01:23.064 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:01:23.064 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:01:23.065 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:01:23.065 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:01:23.065 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:01:23.065 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:01:23.066 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:01:23.066 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:01:23.066 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:01:23.066 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:01:23.066 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:01:23.067 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2022-11-27T16:01:23.067 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2022-11-27T16:01:23.067 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2022-11-27T16:01:23.068 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:01:23.068 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:01:23.068 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:01:23.069 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2022-11-27T16:01:23.069 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.32.1-38.el8 CentOS-BaseOS 99 k 2022-11-27T16:01:23.069 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:1.1.1k-7.el8 CentOS-BaseOS 2.3 M 2022-11-27T16:01:23.069 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 10.19-1.module_el8.6.0+1047+4202cf9a CentOS-AppStream 1.5 M 2022-11-27T16:01:23.070 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 10.19-1.module_el8.6.0+1047+4202cf9a CentOS-AppStream 807 k 2022-11-27T16:01:23.070 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 10.19-1.module_el8.6.0+1047+4202cf9a CentOS-AppStream 5.1 M 2022-11-27T16:01:23.070 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:01:23.070 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:01:23.071 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:01:23.071 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:01:23.071 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:01:23.071 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:01:23.072 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:01:23.072 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2022-11-27T16:01:23.072 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2022-11-27T16:01:23.072 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-initscripts-updown noarch 1:1.40.4-1.el8 CentOS-BaseOS 141 k 2022-11-27T16:01:23.073 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2022-11-27T16:01:23.073 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2022-11-27T16:01:23.073 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2022-11-27T16:01:23.073 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:01:23.074 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:01:23.074 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2022-11-27T16:01:23.074 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2022-11-27T16:01:23.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:01:23.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:01:23.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:01:23.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:01:23.076 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:01:23.076 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:01:23.076 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:01:23.076 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:01:23.077 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T16:01:23.077 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:01:23.077 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T16:01:23.077 INFO:teuthology.orchestra.run.smithi105.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2022-11-27T16:01:23.077 INFO:teuthology.orchestra.run.smithi105.stdout: krb5-devel x86_64 1.18.2-21.el8 CentOS-BaseOS 560 k 2022-11-27T16:01:23.078 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:01:23.078 INFO:teuthology.orchestra.run.smithi105.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2022-11-27T16:01:23.078 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 8.5.0-17.el8 CentOS-BaseOS 645 k 2022-11-27T16:01:23.079 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:01:23.079 INFO:teuthology.orchestra.run.smithi105.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2022-11-27T16:01:23.079 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 8.5.0-17.el8 CentOS-BaseOS 172 k 2022-11-27T16:01:23.079 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T16:01:23.080 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:01:23.080 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T16:01:23.080 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:01:23.080 INFO:teuthology.orchestra.run.smithi105.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2022-11-27T16:01:23.080 INFO:teuthology.orchestra.run.smithi105.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2022-11-27T16:01:23.081 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:01:23.081 INFO:teuthology.orchestra.run.smithi105.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2022-11-27T16:01:23.081 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T16:01:23.081 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T16:01:23.082 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2022-11-27T16:01:23.082 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2022-11-27T16:01:23.082 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2022-11-27T16:01:23.082 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2022-11-27T16:01:23.083 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2022-11-27T16:01:23.083 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:01:23.083 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2022-11-27T16:01:23.083 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:01:23.084 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:01:23.084 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:01:23.084 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:01:23.084 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:01:23.085 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:01:23.085 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T16:01:23.086 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:01:23.086 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:01:23.086 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T16:01:23.086 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:01:23.087 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:01:23.090 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:01:23.090 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2022-11-27T16:01:23.090 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2022-11-27T16:01:23.090 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2022-11-27T16:01:23.091 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2022-11-27T16:01:23.091 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:01:23.091 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:01:23.092 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T16:01:23.092 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T16:01:23.092 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:01:23.092 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:01:23.093 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2022-11-27T16:01:23.093 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2022-11-27T16:01:23.093 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2022-11-27T16:01:23.093 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:01:23.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:01:23.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T16:01:23.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T16:01:23.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2022-11-27T16:01:23.095 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2022-11-27T16:01:23.095 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T16:01:23.095 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T16:01:23.095 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:01:23.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2022-11-27T16:01:23.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:01:23.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:01:23.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2022-11-27T16:01:23.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:01:23.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:01:23.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:01:23.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:01:23.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:01:23.098 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:01:23.098 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:01:23.098 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:01:23.098 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T16:01:23.099 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:01:23.099 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T16:01:23.099 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:01:23.099 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2022-11-27T16:01:23.100 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T16:01:23.100 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-20.el8 CentOS-BaseOS 58 k 2022-11-27T16:01:23.100 INFO:teuthology.orchestra.run.smithi105.stdout:Enabling module streams: 2022-11-27T16:01:23.100 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql 10 2022-11-27T16:01:23.100 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:01:23.101 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2022-11-27T16:01:23.101 INFO:teuthology.orchestra.run.smithi105.stdout:========================================================================================================= 2022-11-27T16:01:23.101 INFO:teuthology.orchestra.run.smithi105.stdout:Install 127 Packages 2022-11-27T16:01:23.101 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 6 Packages 2022-11-27T16:01:23.102 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:01:23.102 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 217 M 2022-11-27T16:01:23.103 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2022-11-27T16:01:23.210 INFO:teuthology.orchestra.run.smithi105.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 48 kB/s | 6.4 kB 00:00 2022-11-27T16:01:23.653 INFO:teuthology.orchestra.run.smithi105.stdout:(2/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 9.0 MB/s | 5.2 MB 00:00 2022-11-27T16:01:23.678 INFO:teuthology.orchestra.run.smithi105.stdout:(3/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 1.8 MB/s | 845 kB 00:00 2022-11-27T16:01:23.720 INFO:teuthology.orchestra.run.smithi105.stdout:(4/133): ceph-immutable-object-cache-18.0.0-594 2.4 MB/s | 165 kB 00:00 2022-11-27T16:01:23.970 INFO:teuthology.orchestra.run.smithi105.stdout:(5/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 6.1 MB/s | 1.5 MB 00:00 2022-11-27T16:01:24.304 INFO:teuthology.orchestra.run.smithi105.stdout:(6/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 12 MB/s | 3.9 MB 00:00 2022-11-27T16:01:24.370 INFO:teuthology.orchestra.run.smithi105.stdout:(7/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 3.3 MB/s | 2.3 MB 00:00 2022-11-27T16:01:24.927 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:01:24.928 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:01:24.939 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.40.0-1.el8.x86_64 is already installed. 2022-11-27T16:01:24.939 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-initscripts-updown-1:1.40.0-1.el8.noarch is already installed. 2022-11-27T16:01:24.940 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-5.18.0-1.el8.x86_64 is already installed. 2022-11-27T16:01:24.940 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.32.1-38.el8.x86_64 is already installed. 2022-11-27T16:01:24.941 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:01:24.943 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:01:24.946 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:01:24.946 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:01:25.059 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2022-11-27T16:01:25.071 INFO:teuthology.orchestra.run.smithi170.stdout:========================================================================================================= 2022-11-27T16:01:25.071 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2022-11-27T16:01:25.072 INFO:teuthology.orchestra.run.smithi170.stdout:========================================================================================================= 2022-11-27T16:01:25.072 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2022-11-27T16:01:25.072 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.0.4-10.el8 CentOS-AppStream 688 k 2022-11-27T16:01:25.072 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:01:25.073 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:01:25.073 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:01:25.073 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:01:25.073 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:01:25.073 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:01:25.074 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:01:25.074 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:01:25.074 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:01:25.074 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:01:25.075 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:01:25.075 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:01:25.075 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:01:25.075 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:01:25.076 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:01:25.076 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.188-3.el8 CentOS-BaseOS 61 k 2022-11-27T16:01:25.076 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.1-9.el8 CentOS-AppStream 320 k 2022-11-27T16:01:25.077 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.112-1.el8 CentOS-BaseOS 19 k 2022-11-27T16:01:25.077 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:01:25.077 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:01:25.078 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:01:25.078 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-25.el8 CentOS-AppStream 709 k 2022-11-27T16:01:25.078 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.32.1-38.el8 CentOS-BaseOS 99 k 2022-11-27T16:01:25.079 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:1.1.1k-7.el8 CentOS-BaseOS 2.3 M 2022-11-27T16:01:25.079 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 10.19-1.module_el8.6.0+1047+4202cf9a CentOS-AppStream 1.5 M 2022-11-27T16:01:25.079 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 10.19-1.module_el8.6.0+1047+4202cf9a CentOS-AppStream 807 k 2022-11-27T16:01:25.080 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 10.19-1.module_el8.6.0+1047+4202cf9a CentOS-AppStream 5.1 M 2022-11-27T16:01:25.080 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:01:25.080 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:01:25.081 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:01:25.081 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:01:25.081 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:01:25.082 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:01:25.082 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:01:25.082 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 CentOS-BaseOS 72 k 2022-11-27T16:01:25.083 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2022-11-27T16:01:25.083 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown noarch 1:1.40.4-1.el8 CentOS-BaseOS 141 k 2022-11-27T16:01:25.084 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-debuginfod-client x86_64 0.188-3.el8 CentOS-BaseOS 75 k 2022-11-27T16:01:25.084 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf x86_64 0.188-3.el8 CentOS-BaseOS 231 k 2022-11-27T16:01:25.084 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs x86_64 0.188-3.el8 CentOS-BaseOS 299 k 2022-11-27T16:01:25.084 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:01:25.085 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:01:25.085 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2022-11-27T16:01:25.085 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.1-7.el8 CentOS-AppStream 713 k 2022-11-27T16:01:25.085 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:01:25.085 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:01:25.086 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:01:25.086 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:01:25.086 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:01:25.086 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:01:25.087 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:01:25.087 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:01:25.088 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.7-16.el8 CentOS-BaseOS 83 k 2022-11-27T16:01:25.088 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:01:25.088 INFO:teuthology.orchestra.run.smithi170.stdout: jq x86_64 1.6-6.el8 CentOS-AppStream 202 k 2022-11-27T16:01:25.088 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 CentOS-BaseOS 48 k 2022-11-27T16:01:25.089 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel x86_64 1.18.2-21.el8 CentOS-BaseOS 560 k 2022-11-27T16:01:25.089 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:01:25.089 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel x86_64 1.45.6-5.el8 CentOS-BaseOS 39 k 2022-11-27T16:01:25.089 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 8.5.0-17.el8 CentOS-BaseOS 645 k 2022-11-27T16:01:25.089 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:01:25.090 INFO:teuthology.orchestra.run.smithi170.stdout: libpq x86_64 13.5-1.el8 CentOS-AppStream 198 k 2022-11-27T16:01:25.090 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 8.5.0-17.el8 CentOS-BaseOS 172 k 2022-11-27T16:01:25.090 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-BaseOS 47 k 2022-11-27T16:01:25.090 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:01:25.091 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 0.11.4-3.el8 CentOS-AppStream 354 k 2022-11-27T16:01:25.091 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:01:25.091 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel x86_64 2.9-6.el8 CentOS-BaseOS 200 k 2022-11-27T16:01:25.091 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel x86_64 2.9-3.el8 CentOS-BaseOS 87 k 2022-11-27T16:01:25.092 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:01:25.092 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel x86_64 0.3.2-2.el8 CentOS-BaseOS 18 k 2022-11-27T16:01:25.092 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2022-11-27T16:01:25.092 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2022-11-27T16:01:25.092 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.15-4.el8 CentOS-AppStream 4.7 M 2022-11-27T16:01:25.093 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads x86_64 0.3.15-4.el8 CentOS-AppStream 4.9 M 2022-11-27T16:01:25.093 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel x86_64 10.32-3.el8 CentOS-BaseOS 605 k 2022-11-27T16:01:25.093 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16 x86_64 10.32-3.el8 CentOS-BaseOS 229 k 2022-11-27T16:01:25.093 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32 x86_64 10.32-3.el8 CentOS-BaseOS 220 k 2022-11-27T16:01:25.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:01:25.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.5.1-7.el8 CentOS-AppStream 4.8 M 2022-11-27T16:01:25.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:01:25.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:01:25.094 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:01:25.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:01:25.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:01:25.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:01:25.095 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-BaseOS 237 k 2022-11-27T16:01:25.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:01:25.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:01:25.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 3.2.1-5.el8 CentOS-BaseOS 559 k 2022-11-27T16:01:25.096 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:01:25.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:01:25.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:01:25.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.10.1-3.el8 CentOS-AppStream 538 k 2022-11-27T16:01:25.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2022-11-27T16:01:25.097 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2022-11-27T16:01:25.098 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-BaseOS 43 k 2022-11-27T16:01:25.098 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:01:25.098 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:01:25.098 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2022-11-27T16:01:25.099 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2022-11-27T16:01:25.099 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:01:25.099 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:01:25.099 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.14.3-10.el8 CentOS-AppStream 3.7 M 2022-11-27T16:01:25.099 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 CentOS-AppStream 225 k 2022-11-27T16:01:25.100 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-BaseOS 155 k 2022-11-27T16:01:25.100 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:01:25.100 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:01:25.100 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2022-11-27T16:01:25.101 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2022-11-27T16:01:25.101 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2022-11-27T16:01:25.101 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2022-11-27T16:01:25.102 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-BaseOS 109 k 2022-11-27T16:01:25.102 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2022-11-27T16:01:25.102 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:01:25.103 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-BaseOS 43 k 2022-11-27T16:01:25.103 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:01:25.104 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:01:25.104 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 CentOS-AppStream 14 M 2022-11-27T16:01:25.104 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:01:25.105 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:01:25.105 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:01:25.106 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:01:25.106 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:01:25.106 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:01:25.107 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:01:25.107 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:01:25.108 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2022-11-27T16:01:25.108 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:01:25.109 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-1.el8 CentOS-AppStream 323 k 2022-11-27T16:01:25.109 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:01:25.110 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-43.el8 CentOS-AppStream 64 k 2022-11-27T16:01:25.110 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el8 CentOS-AppStream 71 k 2022-11-27T16:01:25.110 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-20.el8 CentOS-BaseOS 58 k 2022-11-27T16:01:25.111 INFO:teuthology.orchestra.run.smithi170.stdout:Enabling module streams: 2022-11-27T16:01:25.111 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql 10 2022-11-27T16:01:25.111 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:01:25.112 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2022-11-27T16:01:25.112 INFO:teuthology.orchestra.run.smithi170.stdout:========================================================================================================= 2022-11-27T16:01:25.113 INFO:teuthology.orchestra.run.smithi170.stdout:Install 127 Packages 2022-11-27T16:01:25.113 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 6 Packages 2022-11-27T16:01:25.113 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:01:25.114 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 217 M 2022-11-27T16:01:25.115 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2022-11-27T16:01:25.244 INFO:teuthology.orchestra.run.smithi170.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 41 kB/s | 6.4 kB 00:00 2022-11-27T16:01:25.329 INFO:teuthology.orchestra.run.smithi105.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 11 MB/s | 24 MB 00:02 2022-11-27T16:01:25.421 INFO:teuthology.orchestra.run.smithi105.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 265 kB/s | 24 kB 00:00 2022-11-27T16:01:25.730 INFO:teuthology.orchestra.run.smithi105.stdout:(10/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 12 MB/s | 17 MB 00:01 2022-11-27T16:01:25.847 INFO:teuthology.orchestra.run.smithi105.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 2.2 MB/s | 257 kB 00:00 2022-11-27T16:01:25.889 INFO:teuthology.orchestra.run.smithi105.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 733 kB/s | 30 kB 00:00 2022-11-27T16:01:25.961 INFO:teuthology.orchestra.run.smithi170.stdout:(2/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 5.9 MB/s | 5.2 MB 00:00 2022-11-27T16:01:25.981 INFO:teuthology.orchestra.run.smithi105.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 8.2 MB/s | 767 kB 00:00 2022-11-27T16:01:25.995 INFO:teuthology.orchestra.run.smithi170.stdout:(3/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 1.1 MB/s | 845 kB 00:00 2022-11-27T16:01:26.020 INFO:teuthology.orchestra.run.smithi170.stdout:(4/133): ceph-immutable-object-cache-18.0.0-594 2.8 MB/s | 165 kB 00:00 2022-11-27T16:01:26.048 INFO:teuthology.orchestra.run.smithi105.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 2.0 MB/s | 137 kB 00:00 2022-11-27T16:01:26.139 INFO:teuthology.orchestra.run.smithi105.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.5 MB/s | 139 kB 00:00 2022-11-27T16:01:26.229 INFO:teuthology.orchestra.run.smithi170.stdout:(5/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 7.3 MB/s | 1.5 MB 00:00 2022-11-27T16:01:26.239 INFO:teuthology.orchestra.run.smithi105.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 4.6 MB/s | 474 kB 00:00 2022-11-27T16:01:26.753 INFO:teuthology.orchestra.run.smithi170.stdout:(6/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 7.4 MB/s | 3.9 MB 00:00 2022-11-27T16:01:27.190 INFO:teuthology.orchestra.run.smithi105.stdout:(17/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 6.8 MB/s | 6.5 MB 00:00 2022-11-27T16:01:27.273 INFO:teuthology.orchestra.run.smithi105.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 536 kB/s | 45 kB 00:00 2022-11-27T16:01:27.357 INFO:teuthology.orchestra.run.smithi105.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.3 MB/s | 109 kB 00:00 2022-11-27T16:01:27.396 INFO:teuthology.orchestra.run.smithi170.stdout:(7/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 1.6 MB/s | 2.3 MB 00:01 2022-11-27T16:01:27.449 INFO:teuthology.orchestra.run.smithi105.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 2.0 MB/s | 190 kB 00:00 2022-11-27T16:01:27.541 INFO:teuthology.orchestra.run.smithi105.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 4.0 MB/s | 371 kB 00:00 2022-11-27T16:01:27.633 INFO:teuthology.orchestra.run.smithi105.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 3.8 MB/s | 357 kB 00:00 2022-11-27T16:01:27.725 INFO:teuthology.orchestra.run.smithi105.stdout:(23/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.1 MB/s | 107 kB 00:00 2022-11-27T16:01:27.800 INFO:teuthology.orchestra.run.smithi105.stdout:(24/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.2 MB/s | 94 kB 00:00 2022-11-27T16:01:28.342 INFO:teuthology.orchestra.run.smithi105.stdout:(25/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 6.4 MB/s | 3.5 MB 00:00 2022-11-27T16:01:28.401 INFO:teuthology.orchestra.run.smithi105.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 2.9 MB/s | 175 kB 00:00 2022-11-27T16:01:28.484 INFO:teuthology.orchestra.run.smithi105.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 276 kB/s | 23 kB 00:00 2022-11-27T16:01:28.521 INFO:teuthology.orchestra.run.smithi170.stdout:(8/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86_ 9.5 MB/s | 17 MB 00:01 2022-11-27T16:01:28.580 INFO:teuthology.orchestra.run.smithi170.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 417 kB/s | 24 kB 00:00 2022-11-27T16:01:28.656 INFO:teuthology.orchestra.run.smithi105.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.0 MB/s | 118 kB 00:00 2022-11-27T16:01:28.877 INFO:teuthology.orchestra.run.smithi105.stdout:(29/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 5.9 MB/s | 1.7 MB 00:00 2022-11-27T16:01:29.885 INFO:teuthology.orchestra.run.smithi105.stdout:(30/133): ceph-mgr-diskprediction-local-18.0.0- 7.4 MB/s | 7.4 MB 00:01 2022-11-27T16:01:29.977 INFO:teuthology.orchestra.run.smithi105.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.5 MB/s | 234 kB 00:00 2022-11-27T16:01:30.085 INFO:teuthology.orchestra.run.smithi105.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 437 kB/s | 47 kB 00:00 2022-11-27T16:01:30.169 INFO:teuthology.orchestra.run.smithi105.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 172 kB/s | 14 kB 00:00 2022-11-27T16:01:30.253 INFO:teuthology.orchestra.run.smithi105.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 3.1 MB/s | 263 kB 00:00 2022-11-27T16:01:30.370 INFO:teuthology.orchestra.run.smithi105.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 696 kB/s | 81 kB 00:00 2022-11-27T16:01:30.453 INFO:teuthology.orchestra.run.smithi105.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 185 kB/s | 15 kB 00:00 2022-11-27T16:01:30.520 INFO:teuthology.orchestra.run.smithi105.stdout:(37/133): automake-1.16.1-7.el8.noarch.rpm 10 MB/s | 713 kB 00:00 2022-11-27T16:01:30.554 INFO:teuthology.orchestra.run.smithi105.stdout:(38/133): bison-3.0.4-10.el8.x86_64.rpm 20 MB/s | 688 kB 00:00 2022-11-27T16:01:30.580 INFO:teuthology.orchestra.run.smithi105.stdout:(39/133): flex-2.6.1-9.el8.x86_64.rpm 13 MB/s | 320 kB 00:00 2022-11-27T16:01:30.605 INFO:teuthology.orchestra.run.smithi105.stdout:(40/133): jq-1.6-6.el8.x86_64.rpm 7.9 MB/s | 202 kB 00:00 2022-11-27T16:01:30.656 INFO:teuthology.orchestra.run.smithi105.stdout:(41/133): libpq-13.5-1.el8.x86_64.rpm 3.9 MB/s | 198 kB 00:00 2022-11-27T16:01:30.689 INFO:teuthology.orchestra.run.smithi105.stdout:(42/133): librdkafka-0.11.4-3.el8.x86_64.rpm 10 MB/s | 354 kB 00:00 2022-11-27T16:01:30.723 INFO:teuthology.orchestra.run.smithi105.stdout:(43/133): libtool-2.4.6-25.el8.x86_64.rpm 21 MB/s | 709 kB 00:00 2022-11-27T16:01:30.748 INFO:teuthology.orchestra.run.smithi105.stdout:(44/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 10 MB/s | 259 kB 00:00 2022-11-27T16:01:30.782 INFO:teuthology.orchestra.run.smithi105.stdout:(45/133): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2022-11-27T16:01:30.911 INFO:teuthology.orchestra.run.smithi105.stdout:(46/133): openblas-0.3.15-4.el8.x86_64.rpm 37 MB/s | 4.7 MB 00:00 2022-11-27T16:01:31.024 INFO:teuthology.orchestra.run.smithi105.stdout:(47/133): openblas-threads-0.3.15-4.el8.x86_64. 43 MB/s | 4.9 MB 00:00 2022-11-27T16:01:31.089 INFO:teuthology.orchestra.run.smithi105.stdout:(48/133): postgresql-10.19-1.module_el8.6.0+104 30 MB/s | 1.5 MB 00:00 2022-11-27T16:01:31.108 INFO:teuthology.orchestra.run.smithi105.stdout:(49/133): postgresql-contrib-10.19-1.module_el8 24 MB/s | 807 kB 00:00 2022-11-27T16:01:31.233 INFO:teuthology.orchestra.run.smithi105.stdout:(50/133): postgresql-server-10.19-1.module_el8. 41 MB/s | 5.1 MB 00:00 2022-11-27T16:01:31.330 INFO:teuthology.orchestra.run.smithi170.stdout:(10/133): ceph-common-18.0.0-594.g697b0d99.el8. 3.9 MB/s | 24 MB 00:06 2022-11-27T16:01:31.341 INFO:teuthology.orchestra.run.smithi105.stdout:(51/133): python3-babel-2.5.1-7.el8.noarch.rpm 44 MB/s | 4.8 MB 00:00 2022-11-27T16:01:31.375 INFO:teuthology.orchestra.run.smithi105.stdout:(52/133): python3-jinja2-2.10.1-3.el8.noarch.rp 16 MB/s | 538 kB 00:00 2022-11-27T16:01:31.403 INFO:teuthology.orchestra.run.smithi105.stdout:(53/133): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2022-11-27T16:01:31.439 INFO:teuthology.orchestra.run.smithi170.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 2.3 MB/s | 257 kB 00:00 2022-11-27T16:01:31.499 INFO:teuthology.orchestra.run.smithi105.stdout:(54/133): python3-jsonpointer-1.10-11.el8.noarc 344 kB/s | 20 kB 00:00 2022-11-27T16:01:31.499 INFO:teuthology.orchestra.run.smithi105.stdout:(55/133): python3-mako-1.0.6-13.el8.noarch.rpm 4.5 MB/s | 157 kB 00:00 2022-11-27T16:01:31.514 INFO:teuthology.orchestra.run.smithi170.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 407 kB/s | 30 kB 00:00 2022-11-27T16:01:31.518 INFO:teuthology.orchestra.run.smithi105.stdout:(56/133): python3-markupsafe-0.23-19.el8.x86_64 1.5 MB/s | 39 kB 00:00 2022-11-27T16:01:31.618 INFO:teuthology.orchestra.run.smithi105.stdout:(57/133): python3-numpy-1.14.3-10.el8.x86_64.rp 37 MB/s | 3.7 MB 00:00 2022-11-27T16:01:31.644 INFO:teuthology.orchestra.run.smithi105.stdout:(58/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 8.8 MB/s | 225 kB 00:00 2022-11-27T16:01:31.669 INFO:teuthology.orchestra.run.smithi105.stdout:(59/133): python3-prettytable-0.7.2-14.el8.noar 1.7 MB/s | 44 kB 00:00 2022-11-27T16:01:31.694 INFO:teuthology.orchestra.run.smithi105.stdout:(60/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 4.0 MB/s | 103 kB 00:00 2022-11-27T16:01:31.739 INFO:teuthology.orchestra.run.smithi170.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 3.3 MB/s | 767 kB 00:00 2022-11-27T16:01:31.753 INFO:teuthology.orchestra.run.smithi105.stdout:(61/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2022-11-27T16:01:31.778 INFO:teuthology.orchestra.run.smithi105.stdout:(62/133): python3-pyasn1-modules-0.3.7-6.el8.no 4.3 MB/s | 110 kB 00:00 2022-11-27T16:01:31.803 INFO:teuthology.orchestra.run.smithi105.stdout:(63/133): python3-pytz-2017.2-9.el8.noarch.rpm 2.1 MB/s | 54 kB 00:00 2022-11-27T16:01:31.839 INFO:teuthology.orchestra.run.smithi170.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 2.0 MB/s | 137 kB 00:00 2022-11-27T16:01:31.889 INFO:teuthology.orchestra.run.smithi170.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.6 MB/s | 139 kB 00:00 2022-11-27T16:01:32.040 INFO:teuthology.orchestra.run.smithi170.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 3.1 MB/s | 474 kB 00:00 2022-11-27T16:01:32.104 INFO:teuthology.orchestra.run.smithi105.stdout:(64/133): python3-scipy-1.0.0-21.module_el8.5.0 46 MB/s | 14 MB 00:00 2022-11-27T16:01:32.137 INFO:teuthology.orchestra.run.smithi105.stdout:(65/133): python3-werkzeug-0.12.2-4.el8.noarch. 14 MB/s | 457 kB 00:00 2022-11-27T16:01:32.163 INFO:teuthology.orchestra.run.smithi105.stdout:(66/133): socat-1.7.4.1-1.el8.x86_64.rpm 13 MB/s | 323 kB 00:00 2022-11-27T16:01:32.363 INFO:teuthology.orchestra.run.smithi105.stdout:(67/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 2.3 MB/s | 18 MB 00:07 2022-11-27T16:01:32.388 INFO:teuthology.orchestra.run.smithi105.stdout:(68/133): uuid-1.6.2-43.el8.x86_64.rpm 284 kB/s | 64 kB 00:00 2022-11-27T16:01:32.422 INFO:teuthology.orchestra.run.smithi105.stdout:(69/133): elfutils-libelf-devel-0.188-3.el8.x86 2.1 MB/s | 61 kB 00:00 2022-11-27T16:01:32.439 INFO:teuthology.orchestra.run.smithi105.stdout:(70/133): fuse-2.9.7-16.el8.x86_64.rpm 5.1 MB/s | 83 kB 00:00 2022-11-27T16:01:32.473 INFO:teuthology.orchestra.run.smithi105.stdout:(71/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 762 kB/s | 71 kB 00:00 2022-11-27T16:01:32.474 INFO:teuthology.orchestra.run.smithi105.stdout:(72/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.4 MB/s | 48 kB 00:00 2022-11-27T16:01:32.490 INFO:teuthology.orchestra.run.smithi105.stdout:(73/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.1 MB/s | 19 kB 00:00 2022-11-27T16:01:32.507 INFO:teuthology.orchestra.run.smithi105.stdout:(74/133): libcom_err-devel-1.45.6-5.el8.x86_64. 2.3 MB/s | 39 kB 00:00 2022-11-27T16:01:32.541 INFO:teuthology.orchestra.run.smithi105.stdout:(75/133): krb5-devel-1.18.2-21.el8.x86_64.rpm 6.5 MB/s | 560 kB 00:00 2022-11-27T16:01:32.566 INFO:teuthology.orchestra.run.smithi105.stdout:(76/133): libgfortran-8.5.0-17.el8.x86_64.rpm 11 MB/s | 645 kB 00:00 2022-11-27T16:01:32.612 INFO:teuthology.orchestra.run.smithi105.stdout:(77/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.0 MB/s | 47 kB 00:00 2022-11-27T16:01:32.633 INFO:teuthology.orchestra.run.smithi105.stdout:(78/133): libquadmath-8.5.0-17.el8.x86_64.rpm 1.8 MB/s | 172 kB 00:00 2022-11-27T16:01:32.658 INFO:teuthology.orchestra.run.smithi105.stdout:(79/133): libselinux-devel-2.9-6.el8.x86_64.rpm 4.3 MB/s | 200 kB 00:00 2022-11-27T16:01:32.683 INFO:teuthology.orchestra.run.smithi105.stdout:(80/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2022-11-27T16:01:32.709 INFO:teuthology.orchestra.run.smithi105.stdout:(81/133): libuuid-devel-2.32.1-38.el8.x86_64.rp 1.9 MB/s | 99 kB 00:00 2022-11-27T16:01:32.734 INFO:teuthology.orchestra.run.smithi105.stdout:(82/133): libverto-devel-0.3.2-2.el8.x86_64.rpm 360 kB/s | 18 kB 00:00 2022-11-27T16:01:32.801 INFO:teuthology.orchestra.run.smithi105.stdout:(83/133): openssl-devel-1.1.1k-7.el8.x86_64.rpm 25 MB/s | 2.3 MB 00:00 2022-11-27T16:01:32.827 INFO:teuthology.orchestra.run.smithi105.stdout:(84/133): pcre2-devel-10.32-3.el8.x86_64.rpm 6.4 MB/s | 605 kB 00:00 2022-11-27T16:01:32.852 INFO:teuthology.orchestra.run.smithi105.stdout:(85/133): pcre2-utf16-10.32-3.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2022-11-27T16:01:32.902 INFO:teuthology.orchestra.run.smithi105.stdout:(86/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.6 MB/s | 237 kB 00:00 2022-11-27T16:01:32.927 INFO:teuthology.orchestra.run.smithi105.stdout:(87/133): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.1 MB/s | 220 kB 00:00 2022-11-27T16:01:32.952 INFO:teuthology.orchestra.run.smithi105.stdout:(88/133): python3-jwt-1.6.1-2.el8.noarch.rpm 1.7 MB/s | 43 kB 00:00 2022-11-27T16:01:32.978 INFO:teuthology.orchestra.run.smithi105.stdout:(89/133): python3-cryptography-3.2.1-5.el8.x86_ 7.2 MB/s | 559 kB 00:00 2022-11-27T16:01:33.003 INFO:teuthology.orchestra.run.smithi105.stdout:(90/133): python3-pycparser-2.14-14.el8.noarch. 4.2 MB/s | 109 kB 00:00 2022-11-27T16:01:33.028 INFO:teuthology.orchestra.run.smithi105.stdout:(91/133): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2022-11-27T16:01:33.054 INFO:teuthology.orchestra.run.smithi105.stdout:(92/133): python3-requests-oauthlib-1.0.0-1.el8 848 kB/s | 43 kB 00:00 2022-11-27T16:01:33.079 INFO:teuthology.orchestra.run.smithi105.stdout:(93/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2022-11-27T16:01:33.104 INFO:teuthology.orchestra.run.smithi105.stdout:(94/133): zlib-devel-1.2.11-20.el8.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2022-11-27T16:01:33.238 INFO:teuthology.orchestra.run.smithi105.stdout:(95/133): liboath-2.6.2-3.el8.x86_64.rpm 441 kB/s | 59 kB 00:00 2022-11-27T16:01:33.240 INFO:teuthology.orchestra.run.smithi170.stdout:(17/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 12 MB/s | 55 MB 00:04 2022-11-27T16:01:33.291 INFO:teuthology.orchestra.run.smithi170.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 901 kB/s | 45 kB 00:00 2022-11-27T16:01:33.313 INFO:teuthology.orchestra.run.smithi105.stdout:(96/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2022-11-27T16:01:33.339 INFO:teuthology.orchestra.run.smithi105.stdout:(97/133): libunwind-1.3.1-3.el8.x86_64.rpm 746 kB/s | 75 kB 00:00 2022-11-27T16:01:33.340 INFO:teuthology.orchestra.run.smithi170.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 2.1 MB/s | 109 kB 00:00 2022-11-27T16:01:33.399 INFO:teuthology.orchestra.run.smithi170.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 3.2 MB/s | 190 kB 00:00 2022-11-27T16:01:33.508 INFO:teuthology.orchestra.run.smithi170.stdout:(21/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 4.4 MB/s | 6.5 MB 00:01 2022-11-27T16:01:33.533 INFO:teuthology.orchestra.run.smithi170.stdout:(22/133): python3-rados-18.0.0-594.g697b0d99.el 2.7 MB/s | 371 kB 00:00 2022-11-27T16:01:33.583 INFO:teuthology.orchestra.run.smithi170.stdout:(23/133): python3-rbd-18.0.0-594.g697b0d99.el8. 4.6 MB/s | 357 kB 00:00 2022-11-27T16:01:33.609 INFO:teuthology.orchestra.run.smithi170.stdout:(24/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.4 MB/s | 107 kB 00:00 2022-11-27T16:01:33.626 INFO:teuthology.orchestra.run.smithi170.stdout:(25/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 2.2 MB/s | 94 kB 00:00 2022-11-27T16:01:33.684 INFO:teuthology.orchestra.run.smithi170.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 2.9 MB/s | 175 kB 00:00 2022-11-27T16:01:33.743 INFO:teuthology.orchestra.run.smithi170.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 392 kB/s | 23 kB 00:00 2022-11-27T16:01:33.810 INFO:teuthology.orchestra.run.smithi170.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.7 MB/s | 118 kB 00:00 2022-11-27T16:01:33.868 INFO:teuthology.orchestra.run.smithi170.stdout:(29/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 13 MB/s | 3.5 MB 00:00 2022-11-27T16:01:34.023 INFO:teuthology.orchestra.run.smithi105.stdout:(98/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 6.4 MB/s | 55 MB 00:08 2022-11-27T16:01:34.044 INFO:teuthology.orchestra.run.smithi170.stdout:(30/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 2.8 MB/s | 18 MB 00:06 2022-11-27T16:01:34.098 INFO:teuthology.orchestra.run.smithi105.stdout:(99/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 57 kB/s | 44 kB 00:00 2022-11-27T16:01:34.123 INFO:teuthology.orchestra.run.smithi105.stdout:(100/133): python3-asyncssh-2.7.0-2.el8.noarch. 588 kB/s | 476 kB 00:00 2022-11-27T16:01:34.161 INFO:teuthology.orchestra.run.smithi170.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.0 MB/s | 234 kB 00:00 2022-11-27T16:01:34.199 INFO:teuthology.orchestra.run.smithi105.stdout:(101/133): python3-certifi-2018.10.15-7.el8.noa 217 kB/s | 16 kB 00:00 2022-11-27T16:01:34.203 INFO:teuthology.orchestra.run.smithi170.stdout:(32/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 4.2 MB/s | 1.7 MB 00:00 2022-11-27T16:01:34.224 INFO:teuthology.orchestra.run.smithi105.stdout:(102/133): python3-cachetools-3.1.1-4.el8.noarc 263 kB/s | 33 kB 00:00 2022-11-27T16:01:34.228 INFO:teuthology.orchestra.run.smithi170.stdout:(33/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 705 kB/s | 47 kB 00:00 2022-11-27T16:01:34.249 INFO:teuthology.orchestra.run.smithi105.stdout:(103/133): python3-beautifulsoup4-4.6.3-2.el8.1 820 kB/s | 185 kB 00:00 2022-11-27T16:01:34.253 INFO:teuthology.orchestra.run.smithi170.stdout:(34/133): ceph-prometheus-alerts-18.0.0-594.g69 288 kB/s | 14 kB 00:00 2022-11-27T16:01:34.299 INFO:teuthology.orchestra.run.smithi105.stdout:(104/133): python3-google-auth-1.1.1-10.el8.noa 1.6 MB/s | 81 kB 00:00 2022-11-27T16:01:34.304 INFO:teuthology.orchestra.run.smithi170.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 1.6 MB/s | 81 kB 00:00 2022-11-27T16:01:34.329 INFO:teuthology.orchestra.run.smithi170.stdout:(36/133): ceph-volume-18.0.0-594.g697b0d99.el8. 2.6 MB/s | 263 kB 00:00 2022-11-27T16:01:34.358 INFO:teuthology.orchestra.run.smithi105.stdout:(105/133): python3-cheroot-8.5.2-1.el8.noarch.r 1.1 MB/s | 173 kB 00:00 2022-11-27T16:01:34.372 INFO:teuthology.orchestra.run.smithi170.stdout:(37/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 231 kB/s | 15 kB 00:00 2022-11-27T16:01:34.375 INFO:teuthology.orchestra.run.smithi105.stdout:(106/133): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2022-11-27T16:01:34.400 INFO:teuthology.orchestra.run.smithi105.stdout:(107/133): python3-jaraco-functools-2.0-4.el8.n 421 kB/s | 18 kB 00:00 2022-11-27T16:01:34.434 INFO:teuthology.orchestra.run.smithi105.stdout:(108/133): python3-cherrypy-18.4.0-1.el8.noarch 1.8 MB/s | 384 kB 00:00 2022-11-27T16:01:34.451 INFO:teuthology.orchestra.run.smithi105.stdout:(109/133): python3-logutils-0.3.5-11.el8.noarch 968 kB/s | 49 kB 00:00 2022-11-27T16:01:34.454 INFO:teuthology.orchestra.run.smithi170.stdout:(38/133): ceph-mgr-diskprediction-local-18.0.0- 13 MB/s | 7.4 MB 00:00 2022-11-27T16:01:34.476 INFO:teuthology.orchestra.run.smithi105.stdout:(110/133): python3-more-itertools-7.2.0-3.el8.n 1.4 MB/s | 59 kB 00:00 2022-11-27T16:01:34.493 INFO:teuthology.orchestra.run.smithi105.stdout:(111/133): python3-natsort-7.1.1-2.el8.noarch.r 1.4 MB/s | 60 kB 00:00 2022-11-27T16:01:34.513 INFO:teuthology.orchestra.run.smithi170.stdout:(39/133): automake-1.16.1-7.el8.noarch.rpm 3.8 MB/s | 713 kB 00:00 2022-11-27T16:01:34.535 INFO:teuthology.orchestra.run.smithi105.stdout:(112/133): python3-portend-2.6-1.el8.noarch.rpm 374 kB/s | 16 kB 00:00 2022-11-27T16:01:34.547 INFO:teuthology.orchestra.run.smithi170.stdout:(40/133): jq-1.6-6.el8.x86_64.rpm 6.1 MB/s | 202 kB 00:00 2022-11-27T16:01:34.572 INFO:teuthology.orchestra.run.smithi170.stdout:(41/133): libpq-13.5-1.el8.x86_64.rpm 7.8 MB/s | 198 kB 00:00 2022-11-27T16:01:34.597 INFO:teuthology.orchestra.run.smithi170.stdout:(42/133): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2022-11-27T16:01:34.607 INFO:teuthology.orchestra.run.smithi105.stdout:(113/133): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2022-11-27T16:01:34.627 INFO:teuthology.orchestra.run.smithi105.stdout:(114/133): python3-repoze-lru-0.7-6.el8.noarch. 366 kB/s | 34 kB 00:00 2022-11-27T16:01:34.631 INFO:teuthology.orchestra.run.smithi170.stdout:(43/133): bison-3.0.4-10.el8.x86_64.rpm 2.6 MB/s | 688 kB 00:00 2022-11-27T16:01:34.656 INFO:teuthology.orchestra.run.smithi170.stdout:(44/133): librdkafka-0.11.4-3.el8.x86_64.rpm 4.1 MB/s | 354 kB 00:00 2022-11-27T16:01:34.672 INFO:teuthology.orchestra.run.smithi105.stdout:(115/133): python3-rsa-4.8-1.el8.noarch.rpm 1.4 MB/s | 61 kB 00:00 2022-11-27T16:01:34.681 INFO:teuthology.orchestra.run.smithi170.stdout:(45/133): oniguruma-6.8.2-2.el8.x86_64.rpm 7.3 MB/s | 187 kB 00:00 2022-11-27T16:01:34.707 INFO:teuthology.orchestra.run.smithi170.stdout:(46/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.4 MB/s | 259 kB 00:00 2022-11-27T16:01:34.711 INFO:teuthology.orchestra.run.smithi105.stdout:(116/133): python3-routes-2.4.1-12.el8.noarch.r 1.8 MB/s | 196 kB 00:00 2022-11-27T16:01:34.740 INFO:teuthology.orchestra.run.smithi170.stdout:(47/133): libtool-2.4.6-25.el8.x86_64.rpm 4.9 MB/s | 709 kB 00:00 2022-11-27T16:01:34.744 INFO:teuthology.orchestra.run.smithi105.stdout:(117/133): python3-kubernetes-11.0.0-6.el8.noar 3.2 MB/s | 1.2 MB 00:00 2022-11-27T16:01:34.770 INFO:teuthology.orchestra.run.smithi105.stdout:(118/133): python3-simplegeneric-0.8.1-17.el8.n 192 kB/s | 19 kB 00:00 2022-11-27T16:01:34.795 INFO:teuthology.orchestra.run.smithi105.stdout:(119/133): python3-singledispatch-3.4.0.3-18.el 285 kB/s | 24 kB 00:00 2022-11-27T16:01:34.821 INFO:teuthology.orchestra.run.smithi105.stdout:(120/133): python3-tempora-1.14.1-5.el8.noarch. 581 kB/s | 29 kB 00:00 2022-11-27T16:01:34.824 INFO:teuthology.orchestra.run.smithi170.stdout:(48/133): postgresql-10.19-1.module_el8.6.0+104 18 MB/s | 1.5 MB 00:00 2022-11-27T16:01:34.869 INFO:teuthology.orchestra.run.smithi105.stdout:(121/133): python3-trustme-0.6.0-4.el8.noarch.r 280 kB/s | 27 kB 00:00 2022-11-27T16:01:34.907 INFO:teuthology.orchestra.run.smithi170.stdout:(49/133): openblas-0.3.15-4.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2022-11-27T16:01:34.929 INFO:teuthology.orchestra.run.smithi105.stdout:(122/133): python3-waitress-1.2.1-2.el8.1.noarc 1.8 MB/s | 241 kB 00:00 2022-11-27T16:01:34.954 INFO:teuthology.orchestra.run.smithi105.stdout:(123/133): python3-websocket-client-0.56.0-5.el 715 kB/s | 61 kB 00:00 2022-11-27T16:01:34.966 INFO:teuthology.orchestra.run.smithi170.stdout:(50/133): postgresql-contrib-10.19-1.module_el8 5.6 MB/s | 807 kB 00:00 2022-11-27T16:01:34.980 INFO:teuthology.orchestra.run.smithi105.stdout:(124/133): python3-webob-1.8.5-1.el8.1.noarch.r 1.5 MB/s | 251 kB 00:00 2022-11-27T16:01:35.005 INFO:teuthology.orchestra.run.smithi105.stdout:(125/133): python3-webtest-2.0.33-1.el8.noarch. 1.1 MB/s | 85 kB 00:00 2022-11-27T16:01:35.022 INFO:teuthology.orchestra.run.smithi105.stdout:(126/133): python3-zc-lockfile-2.0-2.el8.noarch 340 kB/s | 23 kB 00:00 2022-11-27T16:01:35.083 INFO:teuthology.orchestra.run.smithi170.stdout:(51/133): postgresql-server-10.19-1.module_el8. 29 MB/s | 5.1 MB 00:00 2022-11-27T16:01:35.116 INFO:teuthology.orchestra.run.smithi170.stdout:(52/133): python3-jinja2-2.10.1-3.el8.noarch.rp 16 MB/s | 538 kB 00:00 2022-11-27T16:01:35.142 INFO:teuthology.orchestra.run.smithi170.stdout:(53/133): python3-jsonpatch-1.21-2.el8.noarch.r 1.0 MB/s | 27 kB 00:00 2022-11-27T16:01:35.167 INFO:teuthology.orchestra.run.smithi170.stdout:(54/133): python3-jsonpointer-1.10-11.el8.noarc 804 kB/s | 20 kB 00:00 2022-11-27T16:01:35.192 INFO:teuthology.orchestra.run.smithi170.stdout:(55/133): python3-mako-1.0.6-13.el8.noarch.rpm 6.1 MB/s | 157 kB 00:00 2022-11-27T16:01:35.218 INFO:teuthology.orchestra.run.smithi170.stdout:(56/133): python3-markupsafe-0.23-19.el8.x86_64 1.5 MB/s | 39 kB 00:00 2022-11-27T16:01:35.293 INFO:teuthology.orchestra.run.smithi170.stdout:(57/133): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2022-11-27T16:01:35.326 INFO:teuthology.orchestra.run.smithi170.stdout:(58/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 6.7 MB/s | 225 kB 00:00 2022-11-27T16:01:35.339 INFO:teuthology.orchestra.run.smithi105.stdout:(127/133): librbd1-18.0.0-594.g697b0d99.el8.x86 11 MB/s | 3.6 MB 00:00 2022-11-27T16:01:35.361 INFO:teuthology.orchestra.run.smithi170.stdout:(59/133): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2022-11-27T16:01:35.406 INFO:teuthology.orchestra.run.smithi105.stdout:(128/133): NetworkManager-initscripts-updown-1. 2.1 MB/s | 141 kB 00:00 2022-11-27T16:01:35.434 INFO:teuthology.orchestra.run.smithi105.stdout:(129/133): elfutils-debuginfod-client-0.188-3.e 3.0 MB/s | 75 kB 00:00 2022-11-27T16:01:35.435 INFO:teuthology.orchestra.run.smithi170.stdout:(60/133): python3-numpy-1.14.3-10.el8.x86_64.rp 18 MB/s | 3.7 MB 00:00 2022-11-27T16:01:35.452 INFO:teuthology.orchestra.run.smithi170.stdout:(61/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.1 MB/s | 103 kB 00:00 2022-11-27T16:01:35.466 INFO:teuthology.orchestra.run.smithi105.stdout:(130/133): elfutils-libelf-0.188-3.el8.x86_64.r 6.9 MB/s | 231 kB 00:00 2022-11-27T16:01:35.478 INFO:teuthology.orchestra.run.smithi170.stdout:(62/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2022-11-27T16:01:35.507 INFO:teuthology.orchestra.run.smithi105.stdout:(131/133): thrift-0.13.0-2.el8.x86_64.rpm 3.3 MB/s | 1.7 MB 00:00 2022-11-27T16:01:35.528 INFO:teuthology.orchestra.run.smithi170.stdout:(63/133): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2022-11-27T16:01:35.532 INFO:teuthology.orchestra.run.smithi105.stdout:(132/133): elfutils-libs-0.188-3.el8.x86_64.rpm 4.4 MB/s | 299 kB 00:00 2022-11-27T16:01:35.545 INFO:teuthology.orchestra.run.smithi170.stdout:(64/133): python3-pytz-2017.2-9.el8.noarch.rpm 806 kB/s | 54 kB 00:00 2022-11-27T16:01:35.579 INFO:teuthology.orchestra.run.smithi170.stdout:(65/133): python3-werkzeug-0.12.2-4.el8.noarch. 13 MB/s | 457 kB 00:00 2022-11-27T16:01:35.612 INFO:teuthology.orchestra.run.smithi170.stdout:(66/133): socat-1.7.4.1-1.el8.x86_64.rpm 9.5 MB/s | 323 kB 00:00 2022-11-27T16:01:35.638 INFO:teuthology.orchestra.run.smithi170.stdout:(67/133): uuid-1.6.2-43.el8.x86_64.rpm 2.5 MB/s | 64 kB 00:00 2022-11-27T16:01:35.705 INFO:teuthology.orchestra.run.smithi170.stdout:(68/133): openblas-threads-0.3.15-4.el8.x86_64. 4.9 MB/s | 4.9 MB 00:00 2022-11-27T16:01:35.730 INFO:teuthology.orchestra.run.smithi170.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 768 kB/s | 71 kB 00:00 2022-11-27T16:01:35.747 INFO:teuthology.orchestra.run.smithi170.stdout:(70/133): elfutils-libelf-devel-0.188-3.el8.x86 1.4 MB/s | 61 kB 00:00 2022-11-27T16:01:35.749 INFO:teuthology.orchestra.run.smithi105.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 4.9 MB/s | 3.6 MB 00:00 2022-11-27T16:01:35.753 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:01:35.754 INFO:teuthology.orchestra.run.smithi105.stdout:Total 17 MB/s | 217 MB 00:12 2022-11-27T16:01:35.797 INFO:teuthology.orchestra.run.smithi170.stdout:(71/133): fuse-2.9.7-16.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2022-11-27T16:01:35.823 INFO:teuthology.orchestra.run.smithi170.stdout:(72/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 638 kB/s | 48 kB 00:00 2022-11-27T16:01:35.840 INFO:teuthology.orchestra.run.smithi170.stdout:(73/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.1 MB/s | 19 kB 00:00 2022-11-27T16:01:35.874 INFO:teuthology.orchestra.run.smithi170.stdout:(74/133): krb5-devel-1.18.2-21.el8.x86_64.rpm 7.3 MB/s | 560 kB 00:00 2022-11-27T16:01:35.890 INFO:teuthology.orchestra.run.smithi170.stdout:(75/133): libcom_err-devel-1.45.6-5.el8.x86_64. 775 kB/s | 39 kB 00:00 2022-11-27T16:01:35.982 INFO:teuthology.orchestra.run.smithi170.stdout:(76/133): libgfortran-8.5.0-17.el8.x86_64.rpm 5.8 MB/s | 645 kB 00:00 2022-11-27T16:01:36.165 INFO:teuthology.orchestra.run.smithi170.stdout:(77/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 255 kB/s | 47 kB 00:00 2022-11-27T16:01:36.191 INFO:teuthology.orchestra.run.smithi170.stdout:(78/133): libquadmath-8.5.0-17.el8.x86_64.rpm 571 kB/s | 172 kB 00:00 2022-11-27T16:01:36.216 INFO:teuthology.orchestra.run.smithi170.stdout:(79/133): libselinux-devel-2.9-6.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2022-11-27T16:01:36.242 INFO:teuthology.orchestra.run.smithi170.stdout:(80/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2022-11-27T16:01:36.267 INFO:teuthology.orchestra.run.smithi170.stdout:(81/133): libuuid-devel-2.32.1-38.el8.x86_64.rp 1.9 MB/s | 99 kB 00:00 2022-11-27T16:01:36.292 INFO:teuthology.orchestra.run.smithi170.stdout:(82/133): libverto-devel-0.3.2-2.el8.x86_64.rpm 359 kB/s | 18 kB 00:00 2022-11-27T16:01:36.392 INFO:teuthology.orchestra.run.smithi170.stdout:(83/133): openssl-devel-1.1.1k-7.el8.x86_64.rpm 19 MB/s | 2.3 MB 00:00 2022-11-27T16:01:36.426 INFO:teuthology.orchestra.run.smithi170.stdout:(84/133): pcre2-devel-10.32-3.el8.x86_64.rpm 4.4 MB/s | 605 kB 00:00 2022-11-27T16:01:36.451 INFO:teuthology.orchestra.run.smithi170.stdout:(85/133): pcre2-utf16-10.32-3.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2022-11-27T16:01:36.485 INFO:teuthology.orchestra.run.smithi170.stdout:(86/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 7.0 MB/s | 237 kB 00:00 2022-11-27T16:01:36.510 INFO:teuthology.orchestra.run.smithi170.stdout:(87/133): pcre2-utf32-10.32-3.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2022-11-27T16:01:36.544 INFO:teuthology.orchestra.run.smithi170.stdout:(88/133): python3-jwt-1.6.1-2.el8.noarch.rpm 1.2 MB/s | 43 kB 00:00 2022-11-27T16:01:36.555 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2022-11-27T16:01:36.578 INFO:teuthology.orchestra.run.smithi170.stdout:(89/133): python3-cryptography-3.2.1-5.el8.x86_ 5.9 MB/s | 559 kB 00:00 2022-11-27T16:01:36.603 INFO:teuthology.orchestra.run.smithi170.stdout:(90/133): python3-pycparser-2.14-14.el8.noarch. 4.3 MB/s | 109 kB 00:00 2022-11-27T16:01:36.628 INFO:teuthology.orchestra.run.smithi170.stdout:(91/133): python3-oauthlib-2.1.0-1.el8.noarch.r 1.8 MB/s | 155 kB 00:00 2022-11-27T16:01:36.671 INFO:teuthology.orchestra.run.smithi170.stdout:(92/133): python3-requests-oauthlib-1.0.0-1.el8 632 kB/s | 43 kB 00:00 2022-11-27T16:01:36.704 INFO:teuthology.orchestra.run.smithi170.stdout:(93/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 959 kB/s | 72 kB 00:00 2022-11-27T16:01:36.729 INFO:teuthology.orchestra.run.smithi170.stdout:(94/133): zlib-devel-1.2.11-20.el8.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2022-11-27T16:01:36.752 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2022-11-27T16:01:36.752 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2022-11-27T16:01:36.863 INFO:teuthology.orchestra.run.smithi170.stdout:(95/133): python3-scipy-1.0.0-21.module_el8.5.0 10 MB/s | 14 MB 00:01 2022-11-27T16:01:36.971 INFO:teuthology.orchestra.run.smithi170.stdout:(96/133): liboath-2.6.2-3.el8.x86_64.rpm 243 kB/s | 59 kB 00:00 2022-11-27T16:01:36.997 INFO:teuthology.orchestra.run.smithi170.stdout:(97/133): libunwind-1.3.1-3.el8.x86_64.rpm 563 kB/s | 75 kB 00:00 2022-11-27T16:01:37.047 INFO:teuthology.orchestra.run.smithi170.stdout:(98/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 872 kB/s | 44 kB 00:00 2022-11-27T16:01:37.072 INFO:teuthology.orchestra.run.smithi170.stdout:(99/133): gperftools-libs-2.7-9.el8.x86_64.rpm 832 kB/s | 306 kB 00:00 2022-11-27T16:01:37.123 INFO:teuthology.orchestra.run.smithi170.stdout:(100/133): python3-cachetools-3.1.1-4.el8.noarc 661 kB/s | 33 kB 00:00 2022-11-27T16:01:37.148 INFO:teuthology.orchestra.run.smithi170.stdout:(101/133): python3-beautifulsoup4-4.6.3-2.el8.1 1.8 MB/s | 185 kB 00:00 2022-11-27T16:01:37.173 INFO:teuthology.orchestra.run.smithi170.stdout:(102/133): python3-asyncssh-2.7.0-2.el8.noarch. 2.3 MB/s | 476 kB 00:00 2022-11-27T16:01:37.190 INFO:teuthology.orchestra.run.smithi170.stdout:(103/133): python3-certifi-2018.10.15-7.el8.noa 243 kB/s | 16 kB 00:00 2022-11-27T16:01:37.249 INFO:teuthology.orchestra.run.smithi170.stdout:(104/133): python3-cheroot-8.5.2-1.el8.noarch.r 1.7 MB/s | 173 kB 00:00 2022-11-27T16:01:37.274 INFO:teuthology.orchestra.run.smithi170.stdout:(105/133): python3-google-auth-1.1.1-10.el8.noa 972 kB/s | 81 kB 00:00 2022-11-27T16:01:37.299 INFO:teuthology.orchestra.run.smithi170.stdout:(106/133): python3-cherrypy-18.4.0-1.el8.noarch 3.0 MB/s | 384 kB 00:00 2022-11-27T16:01:37.316 INFO:teuthology.orchestra.run.smithi170.stdout:(107/133): python3-jaraco-6.2-6.el8.noarch.rpm 166 kB/s | 11 kB 00:00 2022-11-27T16:01:37.333 INFO:teuthology.orchestra.run.smithi170.stdout:(108/133): python3-jaraco-functools-2.0-4.el8.n 298 kB/s | 18 kB 00:00 2022-11-27T16:01:37.358 INFO:teuthology.orchestra.run.smithi170.stdout:(109/133): python3-logutils-0.3.5-11.el8.noarch 1.1 MB/s | 49 kB 00:00 2022-11-27T16:01:37.383 INFO:teuthology.orchestra.run.smithi170.stdout:(110/133): python3-more-itertools-7.2.0-3.el8.n 1.1 MB/s | 59 kB 00:00 2022-11-27T16:01:37.401 INFO:teuthology.orchestra.run.smithi170.stdout:(111/133): python3-natsort-7.1.1-2.el8.noarch.r 1.4 MB/s | 60 kB 00:00 2022-11-27T16:01:37.447 INFO:teuthology.orchestra.run.smithi170.stdout:(112/133): python3-portend-2.6-1.el8.noarch.rpm 374 kB/s | 16 kB 00:00 2022-11-27T16:01:37.484 INFO:teuthology.orchestra.run.smithi170.stdout:(113/133): python3-repoze-lru-0.7-6.el8.noarch. 810 kB/s | 34 kB 00:00 2022-11-27T16:01:37.510 INFO:teuthology.orchestra.run.smithi170.stdout:(114/133): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2022-11-27T16:01:37.552 INFO:teuthology.orchestra.run.smithi170.stdout:(115/133): python3-rsa-4.8-1.el8.noarch.rpm 1.4 MB/s | 61 kB 00:00 2022-11-27T16:01:37.577 INFO:teuthology.orchestra.run.smithi170.stdout:(116/133): python3-routes-2.4.1-12.el8.noarch.r 2.1 MB/s | 196 kB 00:00 2022-11-27T16:01:37.611 INFO:teuthology.orchestra.run.smithi170.stdout:(117/133): python3-kubernetes-11.0.0-6.el8.noar 3.7 MB/s | 1.2 MB 00:00 2022-11-27T16:01:37.636 INFO:teuthology.orchestra.run.smithi170.stdout:(118/133): python3-simplegeneric-0.8.1-17.el8.n 222 kB/s | 19 kB 00:00 2022-11-27T16:01:37.661 INFO:teuthology.orchestra.run.smithi170.stdout:(119/133): python3-singledispatch-3.4.0.3-18.el 285 kB/s | 24 kB 00:00 2022-11-27T16:01:37.686 INFO:teuthology.orchestra.run.smithi170.stdout:(120/133): python3-tempora-1.14.1-5.el8.noarch. 387 kB/s | 29 kB 00:00 2022-11-27T16:01:37.712 INFO:teuthology.orchestra.run.smithi170.stdout:(121/133): python3-trustme-0.6.0-4.el8.noarch.r 551 kB/s | 27 kB 00:00 2022-11-27T16:01:37.762 INFO:teuthology.orchestra.run.smithi170.stdout:(122/133): python3-websocket-client-0.56.0-5.el 1.2 MB/s | 61 kB 00:00 2022-11-27T16:01:37.819 INFO:teuthology.orchestra.run.smithi170.stdout:(123/133): python3-webob-1.8.5-1.el8.1.noarch.r 1.9 MB/s | 251 kB 00:00 2022-11-27T16:01:37.846 INFO:teuthology.orchestra.run.smithi170.stdout:(124/133): python3-waitress-1.2.1-2.el8.1.noarc 1.3 MB/s | 241 kB 00:00 2022-11-27T16:01:37.871 INFO:teuthology.orchestra.run.smithi170.stdout:(125/133): python3-webtest-2.0.33-1.el8.noarch. 783 kB/s | 85 kB 00:00 2022-11-27T16:01:37.897 INFO:teuthology.orchestra.run.smithi170.stdout:(126/133): python3-zc-lockfile-2.0-2.el8.noarch 295 kB/s | 23 kB 00:00 2022-11-27T16:01:38.075 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2022-11-27T16:01:38.076 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2022-11-27T16:01:38.284 INFO:teuthology.orchestra.run.smithi170.stdout:(127/133): librbd1-18.0.0-594.g697b0d99.el8.x86 9.3 MB/s | 3.6 MB 00:00 2022-11-27T16:01:38.305 INFO:teuthology.orchestra.run.smithi170.stdout:(128/133): NetworkManager-initscripts-updown-1. 5.6 MB/s | 141 kB 00:00 2022-11-27T16:01:38.331 INFO:teuthology.orchestra.run.smithi170.stdout:(129/133): elfutils-debuginfod-client-0.188-3.e 3.0 MB/s | 75 kB 00:00 2022-11-27T16:01:38.364 INFO:teuthology.orchestra.run.smithi170.stdout:(130/133): elfutils-libelf-0.188-3.el8.x86_64.r 6.8 MB/s | 231 kB 00:00 2022-11-27T16:01:38.423 INFO:teuthology.orchestra.run.smithi170.stdout:(131/133): thrift-0.13.0-2.el8.x86_64.rpm 3.0 MB/s | 1.7 MB 00:00 2022-11-27T16:01:38.456 INFO:teuthology.orchestra.run.smithi170.stdout:(132/133): elfutils-libs-0.188-3.el8.x86_64.rpm 3.2 MB/s | 299 kB 00:00 2022-11-27T16:01:38.623 INFO:teuthology.orchestra.run.smithi170.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 4.8 MB/s | 3.6 MB 00:00 2022-11-27T16:01:38.628 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:01:38.628 INFO:teuthology.orchestra.run.smithi170.stdout:Total 16 MB/s | 217 MB 00:13 2022-11-27T16:01:39.440 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2022-11-27T16:01:39.638 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2022-11-27T16:01:39.639 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2022-11-27T16:01:40.976 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2022-11-27T16:01:40.979 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2022-11-27T16:01:43.588 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2022-11-27T16:01:43.778 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/139 2022-11-27T16:01:43.993 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/139 2022-11-27T16:01:44.206 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/139 2022-11-27T16:01:44.380 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/139 2022-11-27T16:01:44.811 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/139 2022-11-27T16:01:45.022 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 5/139 2022-11-27T16:01:45.418 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 5/139 2022-11-27T16:01:45.639 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 6/139 2022-11-27T16:01:45.763 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 6/139 2022-11-27T16:01:45.955 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 7/139 2022-11-27T16:01:46.053 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 7/139 2022-11-27T16:01:46.210 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2022-11-27T16:01:46.286 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 8/139 2022-11-27T16:01:46.378 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/139 2022-11-27T16:01:46.455 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/139 2022-11-27T16:01:46.553 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/139 2022-11-27T16:01:46.585 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/139 2022-11-27T16:01:46.755 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/139 2022-11-27T16:01:46.775 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 10/139 2022-11-27T16:01:46.897 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/139 2022-11-27T16:01:46.983 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 11/139 2022-11-27T16:01:47.186 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/139 2022-11-27T16:01:47.284 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/139 2022-11-27T16:01:47.488 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 5/139 2022-11-27T16:01:47.598 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpq-13.5-1.el8.x86_64 13/139 2022-11-27T16:01:47.842 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-10.19-1.module_el8.6.0+1047+4202cf9a. 14/139 2022-11-27T16:01:47.868 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 5/139 2022-11-27T16:01:48.030 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/139 2022-11-27T16:01:48.072 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 6/139 2022-11-27T16:01:48.135 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/139 2022-11-27T16:01:48.196 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 6/139 2022-11-27T16:01:48.313 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/139 2022-11-27T16:01:48.363 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 7/139 2022-11-27T16:01:48.415 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/139 2022-11-27T16:01:48.443 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 7/139 2022-11-27T16:01:48.652 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 17/139 2022-11-27T16:01:48.661 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 8/139 2022-11-27T16:01:48.813 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 9/139 2022-11-27T16:01:48.833 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-20.el8.x86_64 18/139 2022-11-27T16:01:48.892 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 9/139 2022-11-27T16:01:48.963 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-8.5.0-17.el8.x86_64 19/139 2022-11-27T16:01:49.075 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libelf-0.188-3.el8.x86_64 10/139 2022-11-27T16:01:49.079 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libquadmath-8.5.0-17.el8.x86_64 19/139 2022-11-27T16:01:49.221 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-8.5.0-17.el8.x86_64 20/139 2022-11-27T16:01:49.250 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 11/139 2022-11-27T16:01:49.444 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/139 2022-11-27T16:01:49.816 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libgfortran-8.5.0-17.el8.x86_64 20/139 2022-11-27T16:01:49.835 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpq-13.5-1.el8.x86_64 13/139 2022-11-27T16:01:50.005 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.15-4.el8.x86_64 21/139 2022-11-27T16:01:50.067 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-10.19-1.module_el8.6.0+1047+4202cf9a. 14/139 2022-11-27T16:01:50.213 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/139 2022-11-27T16:01:50.316 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/139 2022-11-27T16:01:50.472 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/139 2022-11-27T16:01:50.600 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 21/139 2022-11-27T16:01:50.616 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/139 2022-11-27T16:01:50.810 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 17/139 2022-11-27T16:01:50.821 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 22/139 2022-11-27T16:01:50.963 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-20.el8.x86_64 18/139 2022-11-27T16:01:51.088 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-8.5.0-17.el8.x86_64 19/139 2022-11-27T16:01:51.246 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libquadmath-8.5.0-17.el8.x86_64 19/139 2022-11-27T16:01:51.371 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-8.5.0-17.el8.x86_64 20/139 2022-11-27T16:01:51.505 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 22/139 2022-11-27T16:01:51.847 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/139 2022-11-27T16:01:51.964 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libgfortran-8.5.0-17.el8.x86_64 20/139 2022-11-27T16:01:52.035 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/139 2022-11-27T16:01:52.174 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 25/139 2022-11-27T16:01:52.179 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.15-4.el8.x86_64 21/139 2022-11-27T16:01:52.451 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 26/139 2022-11-27T16:01:52.644 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/139 2022-11-27T16:01:52.774 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/139 2022-11-27T16:01:52.789 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 21/139 2022-11-27T16:01:52.996 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 22/139 2022-11-27T16:01:53.042 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/139 2022-11-27T16:01:53.686 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 22/139 2022-11-27T16:01:53.973 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/139 2022-11-27T16:01:54.021 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/139 2022-11-27T16:01:54.160 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/139 2022-11-27T16:01:54.299 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 25/139 2022-11-27T16:01:54.445 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/139 2022-11-27T16:01:54.543 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 26/139 2022-11-27T16:01:54.674 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/139 2022-11-27T16:01:54.735 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/139 2022-11-27T16:01:54.865 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/139 2022-11-27T16:01:55.120 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/139 2022-11-27T16:01:55.246 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/139 2022-11-27T16:01:56.092 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/139 2022-11-27T16:01:56.444 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/139 2022-11-27T16:01:56.667 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/139 2022-11-27T16:01:56.820 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/139 2022-11-27T16:01:57.018 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/139 2022-11-27T16:01:57.316 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 34/139 2022-11-27T16:01:57.547 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/139 2022-11-27T16:01:57.785 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/139 2022-11-27T16:01:57.968 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 37/139 2022-11-27T16:01:58.204 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 38/139 2022-11-27T16:01:58.417 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 39/139 2022-11-27T16:01:58.597 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/139 2022-11-27T16:01:58.611 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 40/139 2022-11-27T16:01:58.799 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 41/139 2022-11-27T16:01:58.890 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 34/139 2022-11-27T16:01:59.061 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 42/139 2022-11-27T16:01:59.091 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/139 2022-11-27T16:01:59.268 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 43/139 2022-11-27T16:01:59.293 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.8-1.el8.noarch 36/139 2022-11-27T16:01:59.433 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 44/139 2022-11-27T16:01:59.500 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libs-0.188-3.el8.x86_64 37/139 2022-11-27T16:01:59.572 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 45/139 2022-11-27T16:01:59.713 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-debuginfod-client-0.188-3.el8.x86_64 38/139 2022-11-27T16:01:59.854 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 46/139 2022-11-27T16:01:59.884 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 39/139 2022-11-27T16:02:00.094 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 40/139 2022-11-27T16:02:00.099 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 47/139 2022-11-27T16:02:00.264 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 41/139 2022-11-27T16:02:00.315 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 48/139 2022-11-27T16:02:00.500 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 42/139 2022-11-27T16:02:00.582 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/139 2022-11-27T16:02:00.718 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 43/139 2022-11-27T16:02:00.768 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/139 2022-11-27T16:02:00.857 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 44/139 2022-11-27T16:02:00.916 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/139 2022-11-27T16:02:01.013 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 45/139 2022-11-27T16:02:01.135 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 52/139 2022-11-27T16:02:01.260 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 46/139 2022-11-27T16:02:01.303 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-2.6-1.el8.noarch 53/139 2022-11-27T16:02:01.471 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 54/139 2022-11-27T16:02:01.472 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 47/139 2022-11-27T16:02:01.662 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 48/139 2022-11-27T16:02:01.714 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 55/139 2022-11-27T16:02:01.897 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/139 2022-11-27T16:02:01.992 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 56/139 2022-11-27T16:02:02.033 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/139 2022-11-27T16:02:02.188 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/139 2022-11-27T16:02:02.232 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 57/139 2022-11-27T16:02:02.383 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 52/139 2022-11-27T16:02:02.535 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-2.6-1.el8.noarch 53/139 2022-11-27T16:02:02.593 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 58/139 2022-11-27T16:02:02.692 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 54/139 2022-11-27T16:02:02.906 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 59/139 2022-11-27T16:02:02.932 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 55/139 2022-11-27T16:02:03.120 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 60/139 2022-11-27T16:02:03.160 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 56/139 2022-11-27T16:02:03.344 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 57/139 2022-11-27T16:02:03.679 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 58/139 2022-11-27T16:02:03.893 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 61/139 2022-11-27T16:02:03.936 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 59/139 2022-11-27T16:02:04.087 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 62/139 2022-11-27T16:02:04.156 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 60/139 2022-11-27T16:02:04.160 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 62/139 2022-11-27T16:02:04.407 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 63/139 2022-11-27T16:02:04.653 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 64/139 2022-11-27T16:02:04.923 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 61/139 2022-11-27T16:02:05.027 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 65/139 2022-11-27T16:02:05.077 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 62/139 2022-11-27T16:02:05.150 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 62/139 2022-11-27T16:02:05.305 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 66/139 2022-11-27T16:02:05.373 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 63/139 2022-11-27T16:02:05.496 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 67/139 2022-11-27T16:02:05.600 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 64/139 2022-11-27T16:02:05.764 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/139 2022-11-27T16:02:05.938 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 65/139 2022-11-27T16:02:06.015 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 69/139 2022-11-27T16:02:06.164 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 66/139 2022-11-27T16:02:06.354 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 67/139 2022-11-27T16:02:06.606 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/139 2022-11-27T16:02:06.882 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 69/139 2022-11-27T16:02:07.123 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 70/139 2022-11-27T16:02:07.460 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 71/139 2022-11-27T16:02:07.723 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/139 2022-11-27T16:02:07.966 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/139 2022-11-27T16:02:07.995 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 70/139 2022-11-27T16:02:08.283 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 74/139 2022-11-27T16:02:08.344 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 71/139 2022-11-27T16:02:08.560 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/139 2022-11-27T16:02:08.571 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/139 2022-11-27T16:02:08.809 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/139 2022-11-27T16:02:08.829 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 76/139 2022-11-27T16:02:09.072 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 77/139 2022-11-27T16:02:09.109 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 74/139 2022-11-27T16:02:09.406 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/139 2022-11-27T16:02:09.422 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 78/139 2022-11-27T16:02:09.654 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 76/139 2022-11-27T16:02:09.687 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 79/139 2022-11-27T16:02:09.863 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 77/139 2022-11-27T16:02:09.867 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 80/139 2022-11-27T16:02:10.021 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.32.1-38.el8.x86_64 81/139 2022-11-27T16:02:10.167 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 78/139 2022-11-27T16:02:10.318 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 82/139 2022-11-27T16:02:10.362 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 79/139 2022-11-27T16:02:10.525 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 80/139 2022-11-27T16:02:10.648 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 83/139 2022-11-27T16:02:10.666 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.32.1-38.el8.x86_64 81/139 2022-11-27T16:02:10.797 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 84/139 2022-11-27T16:02:10.927 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 82/139 2022-11-27T16:02:11.007 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 85/139 2022-11-27T16:02:11.167 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libselinux-devel-2.9-6.el8.x86_64 83/139 2022-11-27T16:02:11.253 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : krb5-devel-1.18.2-21.el8.x86_64 86/139 2022-11-27T16:02:11.313 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 84/139 2022-11-27T16:02:11.469 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.7-16.el8.x86_64 87/139 2022-11-27T16:02:11.481 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 85/139 2022-11-27T16:02:11.651 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 88/139 2022-11-27T16:02:11.704 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : krb5-devel-1.18.2-21.el8.x86_64 86/139 2022-11-27T16:02:11.878 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.7-16.el8.x86_64 87/139 2022-11-27T16:02:11.959 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-43.el8.x86_64 89/139 2022-11-27T16:02:12.013 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 89/139 2022-11-27T16:02:12.027 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 88/139 2022-11-27T16:02:12.193 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-43.el8.x86_64 89/139 2022-11-27T16:02:12.202 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 90/139 2022-11-27T16:02:12.304 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 89/139 2022-11-27T16:02:12.402 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 91/139 2022-11-27T16:02:12.479 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 90/139 2022-11-27T16:02:12.635 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 91/139 2022-11-27T16:02:16.159 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 92/139 2022-11-27T16:02:16.349 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 92/139 2022-11-27T16:02:16.436 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 92/139 2022-11-27T16:02:16.571 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 92/139 2022-11-27T16:02:17.002 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 92/139 2022-11-27T16:02:17.139 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 92/139 2022-11-27T16:02:17.334 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 93/139 2022-11-27T16:02:17.405 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 93/139 2022-11-27T16:02:17.458 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 93/139 2022-11-27T16:02:17.534 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 94/139 2022-11-27T16:02:17.542 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 93/139 2022-11-27T16:02:17.676 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 94/139 2022-11-27T16:02:36.801 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 94/139 2022-11-27T16:02:36.802 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2022-11-27T16:02:36.802 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2022-11-27T16:02:36.802 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2022-11-27T16:02:36.803 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /run 2022-11-27T16:02:36.803 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2022-11-27T16:02:36.803 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2022-11-27T16:02:36.803 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2022-11-27T16:02:36.804 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2022-11-27T16:02:36.804 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /dev 2022-11-27T16:02:36.804 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:02:37.041 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 94/139 2022-11-27T16:02:37.041 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2022-11-27T16:02:37.042 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2022-11-27T16:02:37.042 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2022-11-27T16:02:37.042 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /run 2022-11-27T16:02:37.043 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2022-11-27T16:02:37.043 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2022-11-27T16:02:37.044 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2022-11-27T16:02:37.044 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2022-11-27T16:02:37.044 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /dev 2022-11-27T16:02:37.044 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:02:37.174 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 95/139 2022-11-27T16:02:37.414 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 95/139 2022-11-27T16:02:38.665 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 95/139 2022-11-27T16:02:38.891 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 96/139 2022-11-27T16:02:38.903 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 95/139 2022-11-27T16:02:39.115 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 96/139 2022-11-27T16:02:39.129 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 96/139 2022-11-27T16:02:39.332 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 97/139 2022-11-27T16:02:39.362 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 96/139 2022-11-27T16:02:39.539 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 97/139 2022-11-27T16:02:39.711 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 97/139 2022-11-27T16:02:39.915 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 98/139 2022-11-27T16:02:39.919 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 97/139 2022-11-27T16:02:39.938 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 98/139 2022-11-27T16:02:40.105 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 98/139 2022-11-27T16:02:40.128 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 98/139 2022-11-27T16:02:40.172 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 99/139 2022-11-27T16:02:40.328 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 99/139 2022-11-27T16:02:40.375 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 100/139 2022-11-27T16:02:40.507 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 100/139 2022-11-27T16:02:40.582 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 101/139 2022-11-27T16:02:40.641 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 101/139 2022-11-27T16:02:40.672 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 101/139 2022-11-27T16:02:40.735 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 101/139 2022-11-27T16:02:40.932 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-6.el8.x86_64 102/139 2022-11-27T16:02:41.022 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : jq-1.6-6.el8.x86_64 102/139 2022-11-27T16:02:41.198 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.1-7.el8.noarch 103/139 2022-11-27T16:02:41.222 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.1-7.el8.noarch 103/139 2022-11-27T16:02:42.708 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 104/139 2022-11-27T16:02:42.736 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 104/139 2022-11-27T16:02:42.856 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 104/139 2022-11-27T16:02:42.926 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 104/139 2022-11-27T16:02:42.969 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 105/139 2022-11-27T16:02:43.021 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 105/139 2022-11-27T16:02:43.133 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 106/139 2022-11-27T16:02:43.218 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 106/139 2022-11-27T16:02:43.350 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 107/139 2022-11-27T16:02:43.448 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 107/139 2022-11-27T16:02:43.556 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 107/139 2022-11-27T16:02:43.655 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 107/139 2022-11-27T16:02:43.887 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 108/139 2022-11-27T16:02:43.998 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 108/139 2022-11-27T16:02:44.846 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 108/139 2022-11-27T16:02:44.959 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 108/139 2022-11-27T16:02:45.054 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 109/139 2022-11-27T16:02:45.122 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 109/139 2022-11-27T16:02:45.147 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 109/139 2022-11-27T16:02:45.219 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 109/139 2022-11-27T16:02:45.520 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 110/139 2022-11-27T16:02:45.582 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 110/139 2022-11-27T16:02:45.720 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 111/139 2022-11-27T16:02:45.754 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 111/139 2022-11-27T16:02:45.840 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 111/139 2022-11-27T16:02:45.874 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 111/139 2022-11-27T16:02:45.929 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 112/139 2022-11-27T16:02:45.940 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 112/139 2022-11-27T16:02:46.080 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 112/139 2022-11-27T16:02:46.091 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 112/139 2022-11-27T16:02:46.133 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 113/139 2022-11-27T16:02:46.274 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 113/139 2022-11-27T16:02:46.289 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-25.el8.x86_64 114/139 2022-11-27T16:02:46.477 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-25.el8.x86_64 114/139 2022-11-27T16:02:51.424 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 114/139 2022-11-27T16:02:51.589 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 114/139 2022-11-27T16:02:53.254 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 115/139 2022-11-27T16:02:53.444 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 116/139 2022-11-27T16:02:53.477 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 116/139 2022-11-27T16:02:53.539 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 115/139 2022-11-27T16:02:53.670 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 117/139 2022-11-27T16:02:53.711 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 117/139 2022-11-27T16:02:53.789 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 116/139 2022-11-27T16:02:53.822 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 116/139 2022-11-27T16:02:53.928 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 118/139 2022-11-27T16:02:54.055 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 117/139 2022-11-27T16:02:54.096 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 117/139 2022-11-27T16:02:54.232 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 118/139 2022-11-27T16:02:54.321 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 118/139 2022-11-27T16:02:54.407 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 119/139 2022-11-27T16:02:54.617 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 119/139 2022-11-27T16:02:54.667 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 118/139 2022-11-27T16:02:54.864 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 119/139 2022-11-27T16:02:54.920 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-10.19-1.module_el8.6.0+1047+4 120/139 2022-11-27T16:02:55.073 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 119/139 2022-11-27T16:02:55.410 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-10.19-1.module_el8.6.0+1047+4 120/139 2022-11-27T16:02:55.755 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 121/139 2022-11-27T16:02:56.056 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:1.1.1k-7.el8.x86_64 122/139 2022-11-27T16:02:56.215 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 123/139 2022-11-27T16:02:56.277 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 121/139 2022-11-27T16:02:56.374 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 124/139 2022-11-27T16:02:56.579 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 125/139 2022-11-27T16:02:56.607 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:1.1.1k-7.el8.x86_64 122/139 2022-11-27T16:02:56.791 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 123/139 2022-11-27T16:02:56.984 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 124/139 2022-11-27T16:02:57.192 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.188-3.el8.x86_64 125/139 2022-11-27T16:02:58.595 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.19-1.module_el8.6.0+1047+42 126/139 2022-11-27T16:02:58.811 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-10.19-1.module_el8.6.0+1047+42 126/139 2022-11-27T16:02:58.828 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-10.19-1.module_el8.6.0+1047+42 126/139 2022-11-27T16:02:58.994 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 127/139 2022-11-27T16:02:59.289 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-10.19-1.module_el8.6.0+1047+42 126/139 2022-11-27T16:02:59.310 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 128/139 2022-11-27T16:02:59.504 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 129/139 2022-11-27T16:02:59.601 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-10.19-1.module_el8.6.0+1047+42 126/139 2022-11-27T16:02:59.618 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-10.19-1.module_el8.6.0+1047+42 126/139 2022-11-27T16:02:59.627 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.4-1.el8 130/139 2022-11-27T16:02:59.637 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.4-1.el8 130/139 2022-11-27T16:02:59.817 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/139 2022-11-27T16:02:59.904 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 127/139 2022-11-27T16:03:00.051 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/139 2022-11-27T16:03:00.094 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 128/139 2022-11-27T16:03:00.138 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/139 2022-11-27T16:03:00.305 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 129/139 2022-11-27T16:03:00.318 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/139 2022-11-27T16:03:00.330 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/139 2022-11-27T16:03:00.479 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.4-1.el8 130/139 2022-11-27T16:03:00.493 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.4-1.el8 130/139 2022-11-27T16:03:00.503 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 134/139 2022-11-27T16:03:00.649 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/139 2022-11-27T16:03:00.719 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 135/139 2022-11-27T16:03:00.719 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 136/139 2022-11-27T16:03:00.886 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/139 2022-11-27T16:03:00.937 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 136/139 2022-11-27T16:03:00.941 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.0-1.el8 137/139 2022-11-27T16:03:00.974 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/139 2022-11-27T16:03:01.047 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.0-1.el8 137/139 2022-11-27T16:03:01.047 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 138/139 2022-11-27T16:03:01.154 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/139 2022-11-27T16:03:01.165 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/139 2022-11-27T16:03:01.312 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 138/139 2022-11-27T16:03:01.380 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.187-4.el8.x86_64 134/139 2022-11-27T16:03:01.484 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 139/139 2022-11-27T16:03:01.598 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.187-4.el8.x86_64 135/139 2022-11-27T16:03:01.599 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 136/139 2022-11-27T16:03:01.847 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 136/139 2022-11-27T16:03:01.851 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.0-1.el8 137/139 2022-11-27T16:03:02.008 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: elfutils-libelf-0.187-4.el8.x86_64 139/139 2022-11-27T16:03:02.009 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/139 2022-11-27T16:03:02.009 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/139 2022-11-27T16:03:02.009 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/139 2022-11-27T16:03:02.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/139 2022-11-27T16:03:02.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/139 2022-11-27T16:03:02.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/139 2022-11-27T16:03:02.010 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/139 2022-11-27T16:03:02.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/139 2022-11-27T16:03:02.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/139 2022-11-27T16:03:02.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/139 2022-11-27T16:03:02.011 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/139 2022-11-27T16:03:02.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/139 2022-11-27T16:03:02.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/139 2022-11-27T16:03:02.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/139 2022-11-27T16:03:02.012 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/139 2022-11-27T16:03:02.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/139 2022-11-27T16:03:02.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/139 2022-11-27T16:03:02.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/139 2022-11-27T16:03:02.013 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/139 2022-11-27T16:03:02.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/139 2022-11-27T16:03:02.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/139 2022-11-27T16:03:02.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/139 2022-11-27T16:03:02.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/139 2022-11-27T16:03:02.014 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/139 2022-11-27T16:03:02.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/139 2022-11-27T16:03:02.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/139 2022-11-27T16:03:02.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/139 2022-11-27T16:03:02.015 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/139 2022-11-27T16:03:02.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/139 2022-11-27T16:03:02.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/139 2022-11-27T16:03:02.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/139 2022-11-27T16:03:02.016 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/139 2022-11-27T16:03:02.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/139 2022-11-27T16:03:02.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/139 2022-11-27T16:03:02.017 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/139 2022-11-27T16:03:02.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/139 2022-11-27T16:03:02.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/139 2022-11-27T16:03:02.018 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/139 2022-11-27T16:03:02.019 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/139 2022-11-27T16:03:02.019 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/139 2022-11-27T16:03:02.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/139 2022-11-27T16:03:02.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : jq-1.6-6.el8.x86_64 42/139 2022-11-27T16:03:02.020 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/139 2022-11-27T16:03:02.021 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/139 2022-11-27T16:03:02.021 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/139 2022-11-27T16:03:02.022 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/139 2022-11-27T16:03:02.022 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/139 2022-11-27T16:03:02.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/139 2022-11-27T16:03:02.023 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/139 2022-11-27T16:03:02.024 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-10.19-1.module_el8.6.0+1047+4202cf9a. 50/139 2022-11-27T16:03:02.024 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-10.19-1.module_el8.6.0+1047+4 51/139 2022-11-27T16:03:02.024 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-10.19-1.module_el8.6.0+1047+42 52/139 2022-11-27T16:03:02.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/139 2022-11-27T16:03:02.025 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/139 2022-11-27T16:03:02.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/139 2022-11-27T16:03:02.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/139 2022-11-27T16:03:02.026 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 57/139 2022-11-27T16:03:02.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/139 2022-11-27T16:03:02.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/139 2022-11-27T16:03:02.027 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/139 2022-11-27T16:03:02.028 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/139 2022-11-27T16:03:02.029 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/139 2022-11-27T16:03:02.029 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/139 2022-11-27T16:03:02.029 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/139 2022-11-27T16:03:02.030 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/139 2022-11-27T16:03:02.030 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/139 2022-11-27T16:03:02.030 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/139 2022-11-27T16:03:02.030 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/139 2022-11-27T16:03:02.031 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/139 2022-11-27T16:03:02.031 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/139 2022-11-27T16:03:02.031 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/139 2022-11-27T16:03:02.032 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/139 2022-11-27T16:03:02.032 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/139 2022-11-27T16:03:02.032 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : krb5-devel-1.18.2-21.el8.x86_64 74/139 2022-11-27T16:03:02.032 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/139 2022-11-27T16:03:02.032 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/139 2022-11-27T16:03:02.033 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-8.5.0-17.el8.x86_64 77/139 2022-11-27T16:03:02.033 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-8.5.0-17.el8.x86_64 78/139 2022-11-27T16:03:02.033 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/139 2022-11-27T16:03:02.033 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/139 2022-11-27T16:03:02.034 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/139 2022-11-27T16:03:02.034 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.32.1-38.el8.x86_64 82/139 2022-11-27T16:03:02.034 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/139 2022-11-27T16:03:02.034 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8.x86_64 84/139 2022-11-27T16:03:02.034 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/139 2022-11-27T16:03:02.035 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/139 2022-11-27T16:03:02.035 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/139 2022-11-27T16:03:02.035 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/139 2022-11-27T16:03:02.035 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/139 2022-11-27T16:03:02.036 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/139 2022-11-27T16:03:02.036 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/139 2022-11-27T16:03:02.036 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/139 2022-11-27T16:03:02.036 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/139 2022-11-27T16:03:02.037 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/139 2022-11-27T16:03:02.037 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-20.el8.x86_64 95/139 2022-11-27T16:03:02.037 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/139 2022-11-27T16:03:02.037 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/139 2022-11-27T16:03:02.037 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/139 2022-11-27T16:03:02.038 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/139 2022-11-27T16:03:02.038 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/139 2022-11-27T16:03:02.038 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/139 2022-11-27T16:03:02.039 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/139 2022-11-27T16:03:02.039 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/139 2022-11-27T16:03:02.039 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/139 2022-11-27T16:03:02.040 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/139 2022-11-27T16:03:02.040 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/139 2022-11-27T16:03:02.040 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/139 2022-11-27T16:03:02.040 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/139 2022-11-27T16:03:02.041 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/139 2022-11-27T16:03:02.041 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/139 2022-11-27T16:03:02.041 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/139 2022-11-27T16:03:02.041 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/139 2022-11-27T16:03:02.042 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/139 2022-11-27T16:03:02.042 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/139 2022-11-27T16:03:02.042 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/139 2022-11-27T16:03:02.042 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/139 2022-11-27T16:03:02.043 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/139 2022-11-27T16:03:02.043 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/139 2022-11-27T16:03:02.043 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/139 2022-11-27T16:03:02.043 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/139 2022-11-27T16:03:02.044 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/139 2022-11-27T16:03:02.044 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/139 2022-11-27T16:03:02.044 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/139 2022-11-27T16:03:02.044 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/139 2022-11-27T16:03:02.045 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/139 2022-11-27T16:03:02.045 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/139 2022-11-27T16:03:02.045 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/139 2022-11-27T16:03:02.046 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 128/139 2022-11-27T16:03:02.046 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/139 2022-11-27T16:03:02.046 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 130/139 2022-11-27T16:03:02.046 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/139 2022-11-27T16:03:02.046 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.4-1.el8 132/139 2022-11-27T16:03:02.047 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.0-1.el8 133/139 2022-11-27T16:03:02.047 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/139 2022-11-27T16:03:02.048 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/139 2022-11-27T16:03:02.048 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/139 2022-11-27T16:03:02.048 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/139 2022-11-27T16:03:02.048 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/139 2022-11-27T16:03:02.049 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.0-1.el8 137/139 2022-11-27T16:03:02.049 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 138/139 2022-11-27T16:03:02.280 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 138/139 2022-11-27T16:03:02.461 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.187-4.el8.x86_64 139/139 2022-11-27T16:03:03.019 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.187-4.el8.x86_64 139/139 2022-11-27T16:03:03.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/139 2022-11-27T16:03:03.020 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/139 2022-11-27T16:03:03.021 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/139 2022-11-27T16:03:03.021 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/139 2022-11-27T16:03:03.022 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/139 2022-11-27T16:03:03.022 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/139 2022-11-27T16:03:03.023 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/139 2022-11-27T16:03:03.023 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/139 2022-11-27T16:03:03.024 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/139 2022-11-27T16:03:03.024 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/139 2022-11-27T16:03:03.025 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/139 2022-11-27T16:03:03.025 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/139 2022-11-27T16:03:03.025 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/139 2022-11-27T16:03:03.026 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/139 2022-11-27T16:03:03.026 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/139 2022-11-27T16:03:03.027 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/139 2022-11-27T16:03:03.027 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/139 2022-11-27T16:03:03.028 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/139 2022-11-27T16:03:03.028 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/139 2022-11-27T16:03:03.029 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/139 2022-11-27T16:03:03.029 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/139 2022-11-27T16:03:03.029 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/139 2022-11-27T16:03:03.030 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/139 2022-11-27T16:03:03.030 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/139 2022-11-27T16:03:03.031 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/139 2022-11-27T16:03:03.031 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/139 2022-11-27T16:03:03.032 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/139 2022-11-27T16:03:03.032 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/139 2022-11-27T16:03:03.033 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/139 2022-11-27T16:03:03.033 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/139 2022-11-27T16:03:03.033 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/139 2022-11-27T16:03:03.034 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/139 2022-11-27T16:03:03.034 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/139 2022-11-27T16:03:03.035 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/139 2022-11-27T16:03:03.035 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/139 2022-11-27T16:03:03.036 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/139 2022-11-27T16:03:03.036 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/139 2022-11-27T16:03:03.037 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/139 2022-11-27T16:03:03.037 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.1-7.el8.noarch 39/139 2022-11-27T16:03:03.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/139 2022-11-27T16:03:03.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/139 2022-11-27T16:03:03.038 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-6.el8.x86_64 42/139 2022-11-27T16:03:03.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpq-13.5-1.el8.x86_64 43/139 2022-11-27T16:03:03.039 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 44/139 2022-11-27T16:03:03.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 45/139 2022-11-27T16:03:03.040 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 46/139 2022-11-27T16:03:03.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 47/139 2022-11-27T16:03:03.041 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 48/139 2022-11-27T16:03:03.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 49/139 2022-11-27T16:03:03.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-10.19-1.module_el8.6.0+1047+4202cf9a. 50/139 2022-11-27T16:03:03.042 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-10.19-1.module_el8.6.0+1047+4 51/139 2022-11-27T16:03:03.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-10.19-1.module_el8.6.0+1047+42 52/139 2022-11-27T16:03:03.043 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 53/139 2022-11-27T16:03:03.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 54/139 2022-11-27T16:03:03.044 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 55/139 2022-11-27T16:03:03.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 56/139 2022-11-27T16:03:03.045 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 57/139 2022-11-27T16:03:03.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 58/139 2022-11-27T16:03:03.046 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 59/139 2022-11-27T16:03:03.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 60/139 2022-11-27T16:03:03.047 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 61/139 2022-11-27T16:03:03.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/139 2022-11-27T16:03:03.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 63/139 2022-11-27T16:03:03.048 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 64/139 2022-11-27T16:03:03.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 65/139 2022-11-27T16:03:03.049 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 66/139 2022-11-27T16:03:03.050 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 67/139 2022-11-27T16:03:03.050 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 68/139 2022-11-27T16:03:03.051 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 69/139 2022-11-27T16:03:03.051 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 70/139 2022-11-27T16:03:03.051 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.188-3.el8.x86_64 71/139 2022-11-27T16:03:03.052 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.7-16.el8.x86_64 72/139 2022-11-27T16:03:03.052 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 73/139 2022-11-27T16:03:03.053 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : krb5-devel-1.18.2-21.el8.x86_64 74/139 2022-11-27T16:03:03.053 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 75/139 2022-11-27T16:03:03.054 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 76/139 2022-11-27T16:03:03.054 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-8.5.0-17.el8.x86_64 77/139 2022-11-27T16:03:03.054 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-8.5.0-17.el8.x86_64 78/139 2022-11-27T16:03:03.055 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 79/139 2022-11-27T16:03:03.055 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libselinux-devel-2.9-6.el8.x86_64 80/139 2022-11-27T16:03:03.056 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 81/139 2022-11-27T16:03:03.056 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.32.1-38.el8.x86_64 82/139 2022-11-27T16:03:03.057 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 83/139 2022-11-27T16:03:03.057 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8.x86_64 84/139 2022-11-27T16:03:03.058 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 85/139 2022-11-27T16:03:03.058 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 86/139 2022-11-27T16:03:03.059 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 87/139 2022-11-27T16:03:03.059 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 88/139 2022-11-27T16:03:03.059 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 89/139 2022-11-27T16:03:03.060 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 90/139 2022-11-27T16:03:03.060 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 91/139 2022-11-27T16:03:03.060 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 92/139 2022-11-27T16:03:03.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 93/139 2022-11-27T16:03:03.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 94/139 2022-11-27T16:03:03.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-20.el8.x86_64 95/139 2022-11-27T16:03:03.061 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 96/139 2022-11-27T16:03:03.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 97/139 2022-11-27T16:03:03.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 98/139 2022-11-27T16:03:03.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 99/139 2022-11-27T16:03:03.062 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 100/139 2022-11-27T16:03:03.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 101/139 2022-11-27T16:03:03.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 102/139 2022-11-27T16:03:03.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 103/139 2022-11-27T16:03:03.063 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 104/139 2022-11-27T16:03:03.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 105/139 2022-11-27T16:03:03.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 106/139 2022-11-27T16:03:03.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 107/139 2022-11-27T16:03:03.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 108/139 2022-11-27T16:03:03.064 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 109/139 2022-11-27T16:03:03.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 110/139 2022-11-27T16:03:03.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 111/139 2022-11-27T16:03:03.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 112/139 2022-11-27T16:03:03.065 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 113/139 2022-11-27T16:03:03.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-2.6-1.el8.noarch 114/139 2022-11-27T16:03:03.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 115/139 2022-11-27T16:03:03.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 116/139 2022-11-27T16:03:03.066 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 117/139 2022-11-27T16:03:03.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 118/139 2022-11-27T16:03:03.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 119/139 2022-11-27T16:03:03.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 120/139 2022-11-27T16:03:03.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 121/139 2022-11-27T16:03:03.067 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 122/139 2022-11-27T16:03:03.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 123/139 2022-11-27T16:03:03.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 124/139 2022-11-27T16:03:03.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 125/139 2022-11-27T16:03:03.068 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 126/139 2022-11-27T16:03:03.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 127/139 2022-11-27T16:03:03.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 128/139 2022-11-27T16:03:03.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 129/139 2022-11-27T16:03:03.069 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 130/139 2022-11-27T16:03:03.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 131/139 2022-11-27T16:03:03.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.4-1.el8 132/139 2022-11-27T16:03:03.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.0-1.el8 133/139 2022-11-27T16:03:03.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.188-3.el8.x86_64 134/139 2022-11-27T16:03:03.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.187-4.el8.x86_64 135/139 2022-11-27T16:03:03.071 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.188-3.el8.x86_64 136/139 2022-11-27T16:03:03.072 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.187-4.el8.x86_64 137/139 2022-11-27T16:03:03.072 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.188-3.el8.x86_64 138/139 2022-11-27T16:03:03.956 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/139 2022-11-27T16:03:03.957 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:03.957 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2022-11-27T16:03:03.957 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-initscripts-updown-1:1.40.4-1.el8.noarch 2022-11-27T16:03:03.958 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2022-11-27T16:03:03.958 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-0.188-3.el8.x86_64 2022-11-27T16:03:03.958 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs-0.188-3.el8.x86_64 2022-11-27T16:03:03.958 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.959 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.959 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2022-11-27T16:03:03.959 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:03:03.959 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:03:03.960 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.960 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.960 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.960 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.960 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.961 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.961 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.961 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.962 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.962 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.962 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.962 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.962 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.963 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.963 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.963 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.964 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.965 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.965 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.965 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.965 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:03.966 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2022-11-27T16:03:03.966 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:03:03.966 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.7-16.el8.x86_64 2022-11-27T16:03:03.966 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:03:03.967 INFO:teuthology.orchestra.run.smithi170.stdout: jq-1.6-6.el8.x86_64 2022-11-27T16:03:03.967 INFO:teuthology.orchestra.run.smithi170.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:03:03.967 INFO:teuthology.orchestra.run.smithi170.stdout: krb5-devel-1.18.2-21.el8.x86_64 2022-11-27T16:03:03.967 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:03:03.967 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.968 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.968 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.968 INFO:teuthology.orchestra.run.smithi170.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2022-11-27T16:03:03.968 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-8.5.0-17.el8.x86_64 2022-11-27T16:03:03.969 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:03:03.969 INFO:teuthology.orchestra.run.smithi170.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:03:03.969 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-8.5.0-17.el8.x86_64 2022-11-27T16:03:03.969 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:03:03.969 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.970 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.970 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:03:03.970 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.970 INFO:teuthology.orchestra.run.smithi170.stdout: libselinux-devel-2.9-6.el8.x86_64 2022-11-27T16:03:03.971 INFO:teuthology.orchestra.run.smithi170.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:03:03.971 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:03:03.971 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:03:03.971 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.32.1-38.el8.x86_64 2022-11-27T16:03:03.972 INFO:teuthology.orchestra.run.smithi170.stdout: libverto-devel-0.3.2-2.el8.x86_64 2022-11-27T16:03:03.972 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:03:03.972 INFO:teuthology.orchestra.run.smithi170.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:03:03.972 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.15-4.el8.x86_64 2022-11-27T16:03:03.973 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-threads-0.3.15-4.el8.x86_64 2022-11-27T16:03:03.973 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:1.1.1k-7.el8.x86_64 2022-11-27T16:03:03.973 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-devel-10.32-3.el8.x86_64 2022-11-27T16:03:03.973 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf16-10.32-3.el8.x86_64 2022-11-27T16:03:03.973 INFO:teuthology.orchestra.run.smithi170.stdout: pcre2-utf32-10.32-3.el8.x86_64 2022-11-27T16:03:03.974 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-10.19-1.module_el8.6.0+1047+4202cf9a.x86_64 2022-11-27T16:03:03.974 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-10.19-1.module_el8.6.0+1047+4202cf9a.x86_64 2022-11-27T16:03:03.974 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-10.19-1.module_el8.6.0+1047+4202cf9a.x86_64 2022-11-27T16:03:03.974 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:03:03.975 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:03:03.975 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:03:03.975 INFO:teuthology.orchestra.run.smithi170.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:03:03.975 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:03:03.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:03:03.976 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:03:03.977 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:03:03.977 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:03:03.977 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:03:03.977 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:03:03.978 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:03:03.978 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:03:03.978 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:03:03.978 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:03:03.979 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:03:03.979 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:03:03.979 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:03:03.979 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:03:03.979 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:03:03.980 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:03:03.980 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:03:03.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:03:03.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:03:03.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:03:03.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:03:03.981 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:03:03.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:03:03.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:03:03.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:03:03.982 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:03:03.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:03:03.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:03:03.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:03:03.983 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:03:03.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:03:03.984 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:03:03.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:03:03.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2022-11-27T16:03:03.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:03:03.985 INFO:teuthology.orchestra.run.smithi170.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:03:03.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:03:03.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:03:03.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:03:03.986 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:03:03.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:03:03.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:03:03.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:03:03.987 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:03:03.988 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.988 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.988 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:03.988 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:03:03.989 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:03:03.989 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:03:03.989 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:03:03.989 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:03:03.990 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-20.el8.x86_64 2022-11-27T16:03:03.990 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:03.990 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2022-11-27T16:03:04.115 DEBUG:teuthology.parallel:result is None 2022-11-27T16:03:05.070 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.187-4.el8.x86_64 139/139 2022-11-27T16:03:05.070 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:05.071 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2022-11-27T16:03:05.071 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-initscripts-updown-1:1.40.4-1.el8.noarch 2022-11-27T16:03:05.071 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.188-3.el8.x86_64 2022-11-27T16:03:05.071 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.188-3.el8.x86_64 2022-11-27T16:03:05.072 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.188-3.el8.x86_64 2022-11-27T16:03:05.072 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.072 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.072 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2022-11-27T16:03:05.073 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:03:05.073 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:03:05.074 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.074 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.075 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.076 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.076 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.076 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.077 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.077 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.077 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.078 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.078 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.078 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.078 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.078 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.079 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.079 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.079 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.080 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.080 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.080 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:03:05.081 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.188-3.el8.x86_64 2022-11-27T16:03:05.081 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:03:05.081 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.7-16.el8.x86_64 2022-11-27T16:03:05.082 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:03:05.082 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-6.el8.x86_64 2022-11-27T16:03:05.083 INFO:teuthology.orchestra.run.smithi105.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:03:05.083 INFO:teuthology.orchestra.run.smithi105.stdout: krb5-devel-1.18.2-21.el8.x86_64 2022-11-27T16:03:05.084 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:03:05.084 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.085 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.085 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.085 INFO:teuthology.orchestra.run.smithi105.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2022-11-27T16:03:05.086 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-8.5.0-17.el8.x86_64 2022-11-27T16:03:05.086 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:03:05.086 INFO:teuthology.orchestra.run.smithi105.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:03:05.087 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-8.5.0-17.el8.x86_64 2022-11-27T16:03:05.087 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:03:05.088 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.088 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.088 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:03:05.089 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.089 INFO:teuthology.orchestra.run.smithi105.stdout: libselinux-devel-2.9-6.el8.x86_64 2022-11-27T16:03:05.090 INFO:teuthology.orchestra.run.smithi105.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:03:05.090 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:03:05.090 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:03:05.090 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.32.1-38.el8.x86_64 2022-11-27T16:03:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: libverto-devel-0.3.2-2.el8.x86_64 2022-11-27T16:03:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:03:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:03:05.091 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.15-4.el8.x86_64 2022-11-27T16:03:05.092 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-threads-0.3.15-4.el8.x86_64 2022-11-27T16:03:05.092 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:1.1.1k-7.el8.x86_64 2022-11-27T16:03:05.092 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-devel-10.32-3.el8.x86_64 2022-11-27T16:03:05.092 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf16-10.32-3.el8.x86_64 2022-11-27T16:03:05.093 INFO:teuthology.orchestra.run.smithi105.stdout: pcre2-utf32-10.32-3.el8.x86_64 2022-11-27T16:03:05.093 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-10.19-1.module_el8.6.0+1047+4202cf9a.x86_64 2022-11-27T16:03:05.093 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-10.19-1.module_el8.6.0+1047+4202cf9a.x86_64 2022-11-27T16:03:05.093 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-10.19-1.module_el8.6.0+1047+4202cf9a.x86_64 2022-11-27T16:03:05.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:03:05.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:03:05.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:03:05.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:03:05.094 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:03:05.095 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.095 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.095 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:03:05.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:03:05.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:03:05.096 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:03:05.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:03:05.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:03:05.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:03:05.097 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:03:05.098 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:03:05.098 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:03:05.098 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:03:05.099 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:03:05.099 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:03:05.099 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:03:05.099 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:03:05.100 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:03:05.100 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:03:05.100 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:03:05.100 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:03:05.101 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:03:05.101 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:03:05.101 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:03:05.101 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:03:05.101 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:03:05.102 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:03:05.102 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:03:05.102 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:03:05.102 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:03:05.103 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:03:05.103 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.103 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.103 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:03:05.104 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:03:05.104 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.104 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:03:05.104 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:03:05.105 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2022-11-27T16:03:05.105 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:03:05.105 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:03:05.105 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:03:05.106 INFO:teuthology.orchestra.run.smithi105.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:03:05.106 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:03:05.106 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:03:05.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:03:05.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:03:05.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:03:05.107 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:03:05.108 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.108 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.108 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:03:05.108 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:03:05.109 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:03:05.109 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:03:05.109 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:03:05.109 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:03:05.110 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-20.el8.x86_64 2022-11-27T16:03:05.110 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:05.111 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2022-11-27T16:03:05.236 DEBUG:teuthology.parallel:result is None 2022-11-27T16:03:05.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo lsb_release -is 2022-11-27T16:03:05.279 INFO:teuthology.orchestra.run.smithi105.stdout:CentOSStream 2022-11-27T16:03:05.281 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:03:05.282 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:03:05.441 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:03:05.464 INFO:teuthology.orchestra.run.smithi105.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:03:05.466 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:03:05.466 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:03:05.467 DEBUG:teuthology.orchestra.run.smithi170:> sudo lsb_release -is 2022-11-27T16:03:05.512 INFO:teuthology.orchestra.run.smithi170.stdout:CentOSStream 2022-11-27T16:03:05.513 DEBUG:teuthology.misc:System to be installed: CentOSStream 2022-11-27T16:03:05.514 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:03:05.671 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:03:05.696 INFO:teuthology.orchestra.run.smithi170.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:03:05.697 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:03:05.697 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:03:05.698 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T16:03:05.698 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:05.699 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:03:05.727 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:05.727 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:03:05.764 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T16:03:05.765 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:05.765 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:03:05.794 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:03:05.862 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:05.863 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:03:05.890 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:03:05.958 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T16:03:05.958 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:05.959 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:03:05.986 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:03:06.052 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:06.053 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:03:06.080 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:03:06.148 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T16:03:06.230 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T16:03:06.231 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /var/log/ceph 2022-11-27T16:03:06.234 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /var/log/ceph 2022-11-27T16:03:06.260 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T16:03:06.261 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:03:06.302 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:03:06.329 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T16:03:06.329 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:03:06.367 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:03:06.428 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-11-27T16:03:06.428 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-11-27T16:03:06.429 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-11-27T16:03:06.429 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:06.429 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:03:06.450 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:03:06.566 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:03:06.594 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:06.594 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:03:06.613 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:03:06.731 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:03:06.765 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T16:03:06.766 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T16:03:06.766 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088633', 'branch': 'main', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/misc}}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088633', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi105.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/l60JzpJHr1FbtS4dRcaBYv0m0w0J/Ltydp031jdNpwQxyOKVdA1Fyhzb6555nk4Tm7iRp9Ib74f/Xf96yzaTppE7ERQt2x3I9UpdIAl5v3FS5S0/4Rznn7rYRMincqoX0mBvZoEpS0niViHHCwqpWDm5ahZlv+N3fQe+qPLfAtBxH0pjQTjY3DqZtbgLW332FOc2hohJxMSLq8I4G1ROLzyE8q+SGfIUoFbIPqqTZzU9CqcUhUNYr4Cbw4l63PKrPoHPhnTe9es3fLQ3UbvUiBMBkgnqSqmH2Hx4CoGgLNtJGdUdaeBEL16K94XIyJGNHUhG3CNZIqxFC5qySENQPCtM2SrJpD2vfaY35SHtpM3pd6BopXmmmob+kG9//aIr7fdgyj2LabB2CbXjxe/vWlNhp/jmGpQfJLV1Fjpp41FlKNFXfJBKxMLBMyaT7RdLwJPHX9qKIyl5SBTsq3rJgd9x6AtWk+CEYdITJbIB/scgfjWt809zTqGkZExsBcc=', 'smithi170.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrA3uoaZw6MOkND0hyZkDHakwB9ke54CwrLdee1vAdOHd6YDdkFnnqQFHn6ohqeIE2mIOONsv8CItMKdOAKxS+oZeAwXxFt6JlPQ1VR/NUUNIvgvJt5XoNaEIZ5vSOHuif8tvKueMvX8pAIgLPyc7gQHrv9CfFSrLVjsDsHjqYqHSXvDKPxrUwky2NnNJ6eAiAVZQeR+JRcN7IaD+Z5SOlP/dKl/PQez5f4f6xI5djYIM2uxVaQLrF/DHPWgzSScTu0GlWtkwhUo6AOTn2VR9nfpz85q1HD6AWy4951ZXkOQ37DM4pY2Ja652X6+nK4THfhA5MZCzpZqBW2rUC0ZECzfixVcIKLC0KROLt45z5DQ2vgNChb5JJsXd2c/8Ls9FGXu6hnYnVAitW+z+x79Y9wdz8zZMaqI4gqHL+VYk7QOftb8ATjRgbv5MAn7d6RP0r0pw4fYN6bJV+y4mdLUy4DuGwHHByWqK0TgBT2ppIPAJjZaOLxTdFoww3BmEnY80='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.20399'} 2022-11-27T16:03:06.767 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:03:06.770 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:03:06.823 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:03:06.828 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:03:06.890 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:06.891 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:03:06.912 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:03:06.912 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2022-11-27T16:03:06.970 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:03:06.971 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:06.971 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 379366 Links: 1 2022-11-27T16:03:06.972 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:06.972 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:06.973 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2022-11-27 16:03:02.683970736 +0000 2022-11-27T16:03:06.973 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2022-11-27 16:00:24.123028079 +0000 2022-11-27T16:03:06.974 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2022-11-27 16:00:24.123028079 +0000 2022-11-27T16:03:06.974 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2022-11-27T16:03:06.975 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:03:07.036 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2022-11-27T16:03:07.037 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2022-11-27T16:03:07.037 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000224238 s, 2.3 MB/s 2022-11-27T16:03:07.038 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:03:07.098 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2022-11-27T16:03:07.158 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:03:07.158 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:07.158 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 380047 Links: 1 2022-11-27T16:03:07.158 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:07.159 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:07.159 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2022-11-27 16:03:02.684970717 +0000 2022-11-27T16:03:07.159 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2022-11-27 16:00:24.666017438 +0000 2022-11-27T16:03:07.160 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2022-11-27 16:00:24.666017438 +0000 2022-11-27T16:03:07.160 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2022-11-27T16:03:07.160 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:03:07.224 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2022-11-27T16:03:07.224 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2022-11-27T16:03:07.224 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000205755 s, 2.5 MB/s 2022-11-27T16:03:07.226 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:03:07.284 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2022-11-27T16:03:07.343 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:03:07.343 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:07.344 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 379591 Links: 1 2022-11-27T16:03:07.344 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:07.344 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:07.345 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2022-11-27 16:03:02.684970717 +0000 2022-11-27T16:03:07.345 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2022-11-27 16:00:25.201006953 +0000 2022-11-27T16:03:07.345 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2022-11-27 16:00:25.201006953 +0000 2022-11-27T16:03:07.346 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2022-11-27T16:03:07.347 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:03:07.408 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2022-11-27T16:03:07.409 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2022-11-27T16:03:07.409 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000194654 s, 2.6 MB/s 2022-11-27T16:03:07.411 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:03:07.470 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2022-11-27T16:03:07.529 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:03:07.529 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:07.530 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 374549 Links: 1 2022-11-27T16:03:07.530 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:07.531 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:07.531 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2022-11-27 16:03:02.684970717 +0000 2022-11-27T16:03:07.532 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2022-11-27 16:00:25.746996252 +0000 2022-11-27T16:03:07.532 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2022-11-27 16:00:25.746996252 +0000 2022-11-27T16:03:07.532 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2022-11-27T16:03:07.533 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:03:07.594 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2022-11-27T16:03:07.595 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2022-11-27T16:03:07.595 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000237275 s, 2.2 MB/s 2022-11-27T16:03:07.596 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:03:07.655 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:07.656 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:07.656 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:03:07.677 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:03:07.677 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2022-11-27T16:03:07.736 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:03:07.737 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:07.737 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 382983 Links: 1 2022-11-27T16:03:07.737 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:07.738 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:07.738 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2022-11-27 16:03:01.679544547 +0000 2022-11-27T16:03:07.738 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2022-11-27 16:00:21.877116854 +0000 2022-11-27T16:03:07.738 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2022-11-27 16:00:21.877116854 +0000 2022-11-27T16:03:07.739 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2022-11-27T16:03:07.739 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:03:07.804 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2022-11-27T16:03:07.805 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2022-11-27T16:03:07.805 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000111294 s, 4.6 MB/s 2022-11-27T16:03:07.807 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:03:07.868 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2022-11-27T16:03:07.928 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:03:07.928 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:07.929 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 381178 Links: 1 2022-11-27T16:03:07.929 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:07.929 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:07.929 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2022-11-27 16:03:01.679544547 +0000 2022-11-27T16:03:07.930 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2022-11-27 16:00:22.409104421 +0000 2022-11-27T16:03:07.930 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2022-11-27 16:00:22.409104421 +0000 2022-11-27T16:03:07.930 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2022-11-27T16:03:07.931 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:03:07.996 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2022-11-27T16:03:07.996 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2022-11-27T16:03:07.997 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000111484 s, 4.6 MB/s 2022-11-27T16:03:07.999 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:03:08.059 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2022-11-27T16:03:08.118 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:03:08.119 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:08.119 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 379525 Links: 1 2022-11-27T16:03:08.120 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:08.120 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:08.120 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2022-11-27 16:03:01.679544547 +0000 2022-11-27T16:03:08.121 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2022-11-27 16:00:22.948091826 +0000 2022-11-27T16:03:08.121 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2022-11-27 16:00:22.948091826 +0000 2022-11-27T16:03:08.121 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2022-11-27T16:03:08.122 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:03:08.185 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2022-11-27T16:03:08.185 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2022-11-27T16:03:08.185 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 7.5895e-05 s, 6.7 MB/s 2022-11-27T16:03:08.187 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:03:08.246 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2022-11-27T16:03:08.305 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:03:08.306 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:03:08.306 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 6h/6d Inode: 383259 Links: 1 2022-11-27T16:03:08.306 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:03:08.307 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:03:08.307 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2022-11-27 16:03:01.679544547 +0000 2022-11-27T16:03:08.307 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2022-11-27 16:00:23.473079557 +0000 2022-11-27T16:03:08.307 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2022-11-27 16:00:23.473079557 +0000 2022-11-27T16:03:08.308 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: - 2022-11-27T16:03:08.308 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:03:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2022-11-27T16:03:08.372 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2022-11-27T16:03:08.373 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.2542e-05 s, 6.2 MB/s 2022-11-27T16:03:08.374 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:03:08.433 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:08.434 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:03:08.434 INFO:tasks.ceph:Generating config... 2022-11-27T16:03:08.436 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T16:03:08.437 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T16:03:08.437 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T16:03:08.437 INFO:tasks.ceph:[client] pid file = 2022-11-27T16:03:08.438 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T16:03:08.438 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T16:03:08.438 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T16:03:08.438 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T16:03:08.439 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T16:03:08.439 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T16:03:08.440 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T16:03:08.440 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T16:03:08.441 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T16:03:08.441 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T16:03:08.442 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T16:03:08.442 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T16:03:08.443 INFO:tasks.ceph:[mgr] debug client = 20 2022-11-27T16:03:08.443 INFO:tasks.ceph:[mgr] debug finisher = 20 2022-11-27T16:03:08.443 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T16:03:08.444 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T16:03:08.444 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T16:03:08.444 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T16:03:08.445 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T16:03:08.445 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T16:03:08.445 INFO:tasks.ceph:[osd] bdev async discard = True 2022-11-27T16:03:08.446 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-11-27T16:03:08.446 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-11-27T16:03:08.446 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T16:03:08.447 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T16:03:08.447 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-11-27T16:03:08.448 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-11-27T16:03:08.448 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T16:03:08.449 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T16:03:08.449 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-11-27T16:03:08.449 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T16:03:08.449 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T16:03:08.450 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T16:03:08.450 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-27T16:03:08.450 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-27T16:03:08.451 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T16:03:08.451 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T16:03:08.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T16:03:08.504 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T16:03:08.507 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T16:03:08.552 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:03:08.620 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.105'), ('mon.b', '172.21.15.170'), ('mon.c', '[v2:172.21.15.170:3301,v1:172.21.15.170:6790]')] 2022-11-27T16:03:08.621 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.105,172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T16:03:08.622 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:08.622 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:03:08.678 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.105 --add b 172.21.15.170 --addv c '[v2:172.21.15.170:3301,v1:172.21.15.170:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:08.766 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:08.766 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: generated fsid 558b439a-8d62-4391-a123-f6ff0625ded9 2022-11-27T16:03:08.766 INFO:teuthology.orchestra.run.smithi105.stdout:setting min_mon_release = pacific 2022-11-27T16:03:08.767 INFO:teuthology.orchestra.run.smithi105.stdout:epoch 0 2022-11-27T16:03:08.767 INFO:teuthology.orchestra.run.smithi105.stdout:fsid 558b439a-8d62-4391-a123-f6ff0625ded9 2022-11-27T16:03:08.767 INFO:teuthology.orchestra.run.smithi105.stdout:last_changed 2022-11-27T16:03:08.765204+0000 2022-11-27T16:03:08.768 INFO:teuthology.orchestra.run.smithi105.stdout:created 2022-11-27T16:03:08.765204+0000 2022-11-27T16:03:08.768 INFO:teuthology.orchestra.run.smithi105.stdout:min_mon_release 16 (pacific) 2022-11-27T16:03:08.768 INFO:teuthology.orchestra.run.smithi105.stdout:election_strategy: 1 2022-11-27T16:03:08.768 INFO:teuthology.orchestra.run.smithi105.stdout:0: [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] mon.a 2022-11-27T16:03:08.768 INFO:teuthology.orchestra.run.smithi105.stdout:1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.b 2022-11-27T16:03:08.769 INFO:teuthology.orchestra.run.smithi105.stdout:2: [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] mon.c 2022-11-27T16:03:08.769 INFO:teuthology.orchestra.run.smithi105.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T16:03:08.770 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:03:08.789 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 558b439a-8d62-4391-a123-f6ff0625ded9... 2022-11-27T16:03:08.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:03:08.838 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:03:08.879 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2022-11-27T16:03:08.879 INFO:teuthology.orchestra.run.smithi105.stdout: chdir = "" 2022-11-27T16:03:08.880 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:03:08.880 INFO:teuthology.orchestra.run.smithi105.stdout: auth supported = cephx 2022-11-27T16:03:08.881 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.881 INFO:teuthology.orchestra.run.smithi105.stdout: filestore xattr use omap = true 2022-11-27T16:03:08.881 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.882 INFO:teuthology.orchestra.run.smithi105.stdout: mon clock drift allowed = 1.000 2022-11-27T16:03:08.882 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.882 INFO:teuthology.orchestra.run.smithi105.stdout: osd crush chooseleaf type = 0 2022-11-27T16:03:08.882 INFO:teuthology.orchestra.run.smithi105.stdout: auth debug = true 2022-11-27T16:03:08.883 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.883 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on old message = true 2022-11-27T16:03:08.883 INFO:teuthology.orchestra.run.smithi105.stdout: ms die on bug = true 2022-11-27T16:03:08.884 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.884 INFO:teuthology.orchestra.run.smithi105.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:03:08.885 INFO:teuthology.orchestra.run.smithi105.stdout: mon pg warn max object skew = 0 2022-11-27T16:03:08.885 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.885 INFO:teuthology.orchestra.run.smithi105.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:03:08.886 INFO:teuthology.orchestra.run.smithi105.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:03:08.886 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.887 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default size = 2 2022-11-27T16:03:08.887 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.888 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow primary affinity = true 2022-11-27T16:03:08.888 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd allow pg remap = true 2022-11-27T16:03:08.888 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:03:08.888 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:03:08.889 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on no sortbitwise = false 2022-11-27T16:03:08.889 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:03:08.889 INFO:teuthology.orchestra.run.smithi105.stdout: mon warn on too few osds = false 2022-11-27T16:03:08.889 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:03:08.889 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:03:08.890 INFO:teuthology.orchestra.run.smithi105.stdout: mon_allow_pool_size_one = true 2022-11-27T16:03:08.890 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.890 INFO:teuthology.orchestra.run.smithi105.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:03:08.890 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.891 INFO:teuthology.orchestra.run.smithi105.stdout: osd default data pool replay window = 5 2022-11-27T16:03:08.891 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.891 INFO:teuthology.orchestra.run.smithi105.stdout: mon allow pool delete = true 2022-11-27T16:03:08.891 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.891 INFO:teuthology.orchestra.run.smithi105.stdout: mon cluster log file level = debug 2022-11-27T16:03:08.892 INFO:teuthology.orchestra.run.smithi105.stdout: debug asserts on shutdown = true 2022-11-27T16:03:08.892 INFO:teuthology.orchestra.run.smithi105.stdout: mon health detail to clog = false 2022-11-27T16:03:08.892 INFO:teuthology.orchestra.run.smithi105.stdout: mon host = "172.21.15.105,172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790]" 2022-11-27T16:03:08.892 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 558b439a-8d62-4391-a123-f6ff0625ded9 2022-11-27T16:03:08.893 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.893 INFO:teuthology.orchestra.run.smithi105.stdout:[osd] 2022-11-27T16:03:08.893 INFO:teuthology.orchestra.run.smithi105.stdout: osd journal size = 100 2022-11-27T16:03:08.893 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.893 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub load threshold = 5.0 2022-11-27T16:03:08.894 INFO:teuthology.orchestra.run.smithi105.stdout: osd scrub max interval = 600 2022-11-27T16:03:08.894 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.894 INFO:teuthology.orchestra.run.smithi105.stdout: osd recover clone overlap = true 2022-11-27T16:03:08.894 INFO:teuthology.orchestra.run.smithi105.stdout: osd recovery max chunk = 1048576 2022-11-27T16:03:08.895 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.895 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug shutdown = true 2022-11-27T16:03:08.895 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug op order = true 2022-11-27T16:03:08.895 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug verify stray on activate = true 2022-11-27T16:03:08.895 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.896 INFO:teuthology.orchestra.run.smithi105.stdout: osd open classes on start = true 2022-11-27T16:03:08.896 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug pg log writeout = true 2022-11-27T16:03:08.896 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.896 INFO:teuthology.orchestra.run.smithi105.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:03:08.897 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.897 INFO:teuthology.orchestra.run.smithi105.stdout: osd map max advance = 10 2022-11-27T16:03:08.897 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.897 INFO:teuthology.orchestra.run.smithi105.stdout: journal zero on create = true 2022-11-27T16:03:08.897 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.898 INFO:teuthology.orchestra.run.smithi105.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:03:08.898 INFO:teuthology.orchestra.run.smithi105.stdout: filestore apply finisher threads = 3 2022-11-27T16:03:08.898 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.898 INFO:teuthology.orchestra.run.smithi105.stdout: bdev debug aio = true 2022-11-27T16:03:08.899 INFO:teuthology.orchestra.run.smithi105.stdout: osd debug misdirected ops = true 2022-11-27T16:03:08.899 INFO:teuthology.orchestra.run.smithi105.stdout: bdev async discard = True 2022-11-27T16:03:08.899 INFO:teuthology.orchestra.run.smithi105.stdout: bdev enable discard = True 2022-11-27T16:03:08.899 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore allocator = bitmap 2022-11-27T16:03:08.899 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore block size = 96636764160 2022-11-27T16:03:08.900 INFO:teuthology.orchestra.run.smithi105.stdout: bluestore fsck on mount = True 2022-11-27T16:03:08.900 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluefs = 1/20 2022-11-27T16:03:08.900 INFO:teuthology.orchestra.run.smithi105.stdout: debug bluestore = 1/20 2022-11-27T16:03:08.900 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2022-11-27T16:03:08.900 INFO:teuthology.orchestra.run.smithi105.stdout: debug osd = 20 2022-11-27T16:03:08.901 INFO:teuthology.orchestra.run.smithi105.stdout: debug rocksdb = 4/10 2022-11-27T16:03:08.901 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:03:08.901 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd full ratio = 0.9 2022-11-27T16:03:08.901 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:03:08.902 INFO:teuthology.orchestra.run.smithi105.stdout: osd failsafe full ratio = 0.95 2022-11-27T16:03:08.902 INFO:teuthology.orchestra.run.smithi105.stdout: osd objectstore = bluestore 2022-11-27T16:03:08.902 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2022-11-27T16:03:08.902 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.902 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr] 2022-11-27T16:03:08.903 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2022-11-27T16:03:08.903 INFO:teuthology.orchestra.run.smithi105.stdout: debug mgr = 20 2022-11-27T16:03:08.903 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2022-11-27T16:03:08.903 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2022-11-27T16:03:08.904 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:03:08.904 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:03:08.904 INFO:teuthology.orchestra.run.smithi105.stdout: mgr/telemetry/nag = false 2022-11-27T16:03:08.904 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2022-11-27T16:03:08.904 INFO:teuthology.orchestra.run.smithi105.stdout: debug finisher = 20 2022-11-27T16:03:08.905 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.905 INFO:teuthology.orchestra.run.smithi105.stdout:[mon] 2022-11-27T16:03:08.905 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2022-11-27T16:03:08.905 INFO:teuthology.orchestra.run.smithi105.stdout: debug mon = 20 2022-11-27T16:03:08.906 INFO:teuthology.orchestra.run.smithi105.stdout: debug paxos = 20 2022-11-27T16:03:08.906 INFO:teuthology.orchestra.run.smithi105.stdout: debug auth = 20 2022-11-27T16:03:08.906 INFO:teuthology.orchestra.run.smithi105.stdout: mon data avail warn = 5 2022-11-27T16:03:08.906 INFO:teuthology.orchestra.run.smithi105.stdout: mon mgr mkfs grace = 240 2022-11-27T16:03:08.907 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:03:08.907 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd reporter subtree level = osd 2022-11-27T16:03:08.907 INFO:teuthology.orchestra.run.smithi105.stdout: mon osd prime pg temp = true 2022-11-27T16:03:08.907 INFO:teuthology.orchestra.run.smithi105.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:03:08.908 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.908 INFO:teuthology.orchestra.run.smithi105.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:03:08.908 INFO:teuthology.orchestra.run.smithi105.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:03:08.908 INFO:teuthology.orchestra.run.smithi105.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:03:08.908 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.909 INFO:teuthology.orchestra.run.smithi105.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:03:08.909 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:03:08.909 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:03:08.909 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.909 INFO:teuthology.orchestra.run.smithi105.stdout: # 1m isn't quite enough 2022-11-27T16:03:08.910 INFO:teuthology.orchestra.run.smithi105.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:03:08.910 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.910 INFO:teuthology.orchestra.run.smithi105.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:03:08.910 INFO:teuthology.orchestra.run.smithi105.stdout: mon op complaint time = 120 2022-11-27T16:03:08.911 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:08.911 INFO:teuthology.orchestra.run.smithi105.stdout:[client] 2022-11-27T16:03:08.911 INFO:teuthology.orchestra.run.smithi105.stdout: rgw cache enabled = true 2022-11-27T16:03:08.911 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable ops log = true 2022-11-27T16:03:08.911 INFO:teuthology.orchestra.run.smithi105.stdout: rgw enable usage log = true 2022-11-27T16:03:08.912 INFO:teuthology.orchestra.run.smithi105.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:03:08.912 INFO:teuthology.orchestra.run.smithi105.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:03:08.912 INFO:teuthology.orchestra.run.smithi105.stdout: client mount timeout = 600 2022-11-27T16:03:08.912 INFO:teuthology.orchestra.run.smithi105.stdout: debug client = 20 2022-11-27T16:03:08.913 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2022-11-27T16:03:08.913 INFO:teuthology.orchestra.run.smithi105.stdout: pid file = "" 2022-11-27T16:03:08.913 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2022-11-27T16:03:08.913 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2022-11-27T16:03:08.914 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.a] 2022-11-27T16:03:08.914 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.b] 2022-11-27T16:03:08.914 INFO:teuthology.orchestra.run.smithi105.stdout:[mon.c] 2022-11-27T16:03:08.914 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2022-11-27T16:03:08.915 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2022-11-27T16:03:08.916 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2022-11-27T16:03:08.916 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2022-11-27T16:03:08.916 INFO:teuthology.orchestra.run.smithi105.stdout:[mds] 2022-11-27T16:03:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds = 20 2022-11-27T16:03:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: debug mds balancer = 20 2022-11-27T16:03:08.917 INFO:teuthology.orchestra.run.smithi105.stdout: debug ms = 1 2022-11-27T16:03:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug frag = True 2022-11-27T16:03:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: mds debug scatterstat = True 2022-11-27T16:03:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: mds op complaint time = 180 2022-11-27T16:03:08.918 INFO:teuthology.orchestra.run.smithi105.stdout: mds verify scatter = True 2022-11-27T16:03:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: osd op complaint time = 180 2022-11-27T16:03:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: rados mon op timeout = 900 2022-11-27T16:03:08.919 INFO:teuthology.orchestra.run.smithi105.stdout: rados osd op timeout = 900 2022-11-27T16:03:08.922 INFO:teuthology.orchestra.run.smithi170.stdout:[global] 2022-11-27T16:03:08.922 INFO:teuthology.orchestra.run.smithi170.stdout: chdir = "" 2022-11-27T16:03:08.922 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:03:08.923 INFO:teuthology.orchestra.run.smithi170.stdout: auth supported = cephx 2022-11-27T16:03:08.923 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.923 INFO:teuthology.orchestra.run.smithi170.stdout: filestore xattr use omap = true 2022-11-27T16:03:08.923 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.924 INFO:teuthology.orchestra.run.smithi170.stdout: mon clock drift allowed = 1.000 2022-11-27T16:03:08.924 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.924 INFO:teuthology.orchestra.run.smithi170.stdout: osd crush chooseleaf type = 0 2022-11-27T16:03:08.924 INFO:teuthology.orchestra.run.smithi170.stdout: auth debug = true 2022-11-27T16:03:08.924 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.925 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on old message = true 2022-11-27T16:03:08.925 INFO:teuthology.orchestra.run.smithi170.stdout: ms die on bug = true 2022-11-27T16:03:08.925 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.925 INFO:teuthology.orchestra.run.smithi170.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:03:08.926 INFO:teuthology.orchestra.run.smithi170.stdout: mon pg warn max object skew = 0 2022-11-27T16:03:08.926 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.926 INFO:teuthology.orchestra.run.smithi170.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:03:08.926 INFO:teuthology.orchestra.run.smithi170.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:03:08.927 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.927 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default size = 2 2022-11-27T16:03:08.927 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.928 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow primary affinity = true 2022-11-27T16:03:08.928 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd allow pg remap = true 2022-11-27T16:03:08.928 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:03:08.929 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:03:08.929 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on no sortbitwise = false 2022-11-27T16:03:08.929 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:03:08.929 INFO:teuthology.orchestra.run.smithi170.stdout: mon warn on too few osds = false 2022-11-27T16:03:08.930 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:03:08.930 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:03:08.930 INFO:teuthology.orchestra.run.smithi170.stdout: mon_allow_pool_size_one = true 2022-11-27T16:03:08.931 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.931 INFO:teuthology.orchestra.run.smithi170.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:03:08.931 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.932 INFO:teuthology.orchestra.run.smithi170.stdout: osd default data pool replay window = 5 2022-11-27T16:03:08.932 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.932 INFO:teuthology.orchestra.run.smithi170.stdout: mon allow pool delete = true 2022-11-27T16:03:08.932 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.933 INFO:teuthology.orchestra.run.smithi170.stdout: mon cluster log file level = debug 2022-11-27T16:03:08.933 INFO:teuthology.orchestra.run.smithi170.stdout: debug asserts on shutdown = true 2022-11-27T16:03:08.933 INFO:teuthology.orchestra.run.smithi170.stdout: mon health detail to clog = false 2022-11-27T16:03:08.933 INFO:teuthology.orchestra.run.smithi170.stdout: mon host = "172.21.15.105,172.21.15.170,[v2:172.21.15.170:3301,v1:172.21.15.170:6790]" 2022-11-27T16:03:08.934 INFO:teuthology.orchestra.run.smithi170.stdout: fsid = 558b439a-8d62-4391-a123-f6ff0625ded9 2022-11-27T16:03:08.934 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.934 INFO:teuthology.orchestra.run.smithi170.stdout:[osd] 2022-11-27T16:03:08.934 INFO:teuthology.orchestra.run.smithi170.stdout: osd journal size = 100 2022-11-27T16:03:08.934 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.935 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub load threshold = 5.0 2022-11-27T16:03:08.935 INFO:teuthology.orchestra.run.smithi170.stdout: osd scrub max interval = 600 2022-11-27T16:03:08.935 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.935 INFO:teuthology.orchestra.run.smithi170.stdout: osd recover clone overlap = true 2022-11-27T16:03:08.936 INFO:teuthology.orchestra.run.smithi170.stdout: osd recovery max chunk = 1048576 2022-11-27T16:03:08.936 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.936 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug shutdown = true 2022-11-27T16:03:08.937 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug op order = true 2022-11-27T16:03:08.937 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug verify stray on activate = true 2022-11-27T16:03:08.937 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.937 INFO:teuthology.orchestra.run.smithi170.stdout: osd open classes on start = true 2022-11-27T16:03:08.938 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug pg log writeout = true 2022-11-27T16:03:08.938 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.938 INFO:teuthology.orchestra.run.smithi170.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:03:08.938 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.939 INFO:teuthology.orchestra.run.smithi170.stdout: osd map max advance = 10 2022-11-27T16:03:08.939 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.939 INFO:teuthology.orchestra.run.smithi170.stdout: journal zero on create = true 2022-11-27T16:03:08.939 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.940 INFO:teuthology.orchestra.run.smithi170.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:03:08.940 INFO:teuthology.orchestra.run.smithi170.stdout: filestore apply finisher threads = 3 2022-11-27T16:03:08.940 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.940 INFO:teuthology.orchestra.run.smithi170.stdout: bdev debug aio = true 2022-11-27T16:03:08.941 INFO:teuthology.orchestra.run.smithi170.stdout: osd debug misdirected ops = true 2022-11-27T16:03:08.941 INFO:teuthology.orchestra.run.smithi170.stdout: bdev async discard = True 2022-11-27T16:03:08.941 INFO:teuthology.orchestra.run.smithi170.stdout: bdev enable discard = True 2022-11-27T16:03:08.942 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore allocator = bitmap 2022-11-27T16:03:08.942 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore block size = 96636764160 2022-11-27T16:03:08.942 INFO:teuthology.orchestra.run.smithi170.stdout: bluestore fsck on mount = True 2022-11-27T16:03:08.943 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluefs = 1/20 2022-11-27T16:03:08.943 INFO:teuthology.orchestra.run.smithi170.stdout: debug bluestore = 1/20 2022-11-27T16:03:08.943 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2022-11-27T16:03:08.943 INFO:teuthology.orchestra.run.smithi170.stdout: debug osd = 20 2022-11-27T16:03:08.944 INFO:teuthology.orchestra.run.smithi170.stdout: debug rocksdb = 4/10 2022-11-27T16:03:08.944 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:03:08.944 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd full ratio = 0.9 2022-11-27T16:03:08.944 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:03:08.945 INFO:teuthology.orchestra.run.smithi170.stdout: osd failsafe full ratio = 0.95 2022-11-27T16:03:08.945 INFO:teuthology.orchestra.run.smithi170.stdout: osd objectstore = bluestore 2022-11-27T16:03:08.945 INFO:teuthology.orchestra.run.smithi170.stdout: osd op complaint time = 180 2022-11-27T16:03:08.946 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.946 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr] 2022-11-27T16:03:08.946 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2022-11-27T16:03:08.946 INFO:teuthology.orchestra.run.smithi170.stdout: debug mgr = 20 2022-11-27T16:03:08.946 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2022-11-27T16:03:08.947 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2022-11-27T16:03:08.947 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:03:08.947 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:03:08.947 INFO:teuthology.orchestra.run.smithi170.stdout: mgr/telemetry/nag = false 2022-11-27T16:03:08.948 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2022-11-27T16:03:08.948 INFO:teuthology.orchestra.run.smithi170.stdout: debug finisher = 20 2022-11-27T16:03:08.948 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.948 INFO:teuthology.orchestra.run.smithi170.stdout:[mon] 2022-11-27T16:03:08.949 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2022-11-27T16:03:08.949 INFO:teuthology.orchestra.run.smithi170.stdout: debug mon = 20 2022-11-27T16:03:08.949 INFO:teuthology.orchestra.run.smithi170.stdout: debug paxos = 20 2022-11-27T16:03:08.949 INFO:teuthology.orchestra.run.smithi170.stdout: debug auth = 20 2022-11-27T16:03:08.950 INFO:teuthology.orchestra.run.smithi170.stdout: mon data avail warn = 5 2022-11-27T16:03:08.950 INFO:teuthology.orchestra.run.smithi170.stdout: mon mgr mkfs grace = 240 2022-11-27T16:03:08.950 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:03:08.950 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd reporter subtree level = osd 2022-11-27T16:03:08.950 INFO:teuthology.orchestra.run.smithi170.stdout: mon osd prime pg temp = true 2022-11-27T16:03:08.951 INFO:teuthology.orchestra.run.smithi170.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:03:08.951 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.951 INFO:teuthology.orchestra.run.smithi170.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:03:08.951 INFO:teuthology.orchestra.run.smithi170.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:03:08.952 INFO:teuthology.orchestra.run.smithi170.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:03:08.952 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.952 INFO:teuthology.orchestra.run.smithi170.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:03:08.952 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:03:08.953 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:03:08.953 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.953 INFO:teuthology.orchestra.run.smithi170.stdout: # 1m isn't quite enough 2022-11-27T16:03:08.953 INFO:teuthology.orchestra.run.smithi170.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:03:08.954 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.954 INFO:teuthology.orchestra.run.smithi170.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:03:08.954 INFO:teuthology.orchestra.run.smithi170.stdout: mon op complaint time = 120 2022-11-27T16:03:08.954 INFO:teuthology.orchestra.run.smithi170.stdout: 2022-11-27T16:03:08.954 INFO:teuthology.orchestra.run.smithi170.stdout:[client] 2022-11-27T16:03:08.955 INFO:teuthology.orchestra.run.smithi170.stdout: rgw cache enabled = true 2022-11-27T16:03:08.955 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable ops log = true 2022-11-27T16:03:08.955 INFO:teuthology.orchestra.run.smithi170.stdout: rgw enable usage log = true 2022-11-27T16:03:08.955 INFO:teuthology.orchestra.run.smithi170.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:03:08.956 INFO:teuthology.orchestra.run.smithi170.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:03:08.956 INFO:teuthology.orchestra.run.smithi170.stdout: client mount timeout = 600 2022-11-27T16:03:08.956 INFO:teuthology.orchestra.run.smithi170.stdout: debug client = 20 2022-11-27T16:03:08.956 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2022-11-27T16:03:08.957 INFO:teuthology.orchestra.run.smithi170.stdout: pid file = "" 2022-11-27T16:03:08.957 INFO:teuthology.orchestra.run.smithi170.stdout: rados mon op timeout = 900 2022-11-27T16:03:08.957 INFO:teuthology.orchestra.run.smithi170.stdout: rados osd op timeout = 900 2022-11-27T16:03:08.957 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.a] 2022-11-27T16:03:08.958 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.b] 2022-11-27T16:03:08.958 INFO:teuthology.orchestra.run.smithi170.stdout:[mon.c] 2022-11-27T16:03:08.958 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2022-11-27T16:03:08.958 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2022-11-27T16:03:08.958 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2022-11-27T16:03:08.959 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2022-11-27T16:03:08.959 INFO:teuthology.orchestra.run.smithi170.stdout:[mds] 2022-11-27T16:03:08.959 INFO:teuthology.orchestra.run.smithi170.stdout: debug mds = 20 2022-11-27T16:03:08.959 INFO:teuthology.orchestra.run.smithi170.stdout: debug mds balancer = 20 2022-11-27T16:03:08.960 INFO:teuthology.orchestra.run.smithi170.stdout: debug ms = 1 2022-11-27T16:03:08.960 INFO:teuthology.orchestra.run.smithi170.stdout: mds debug frag = True 2022-11-27T16:03:08.960 INFO:teuthology.orchestra.run.smithi170.stdout: mds debug scatterstat = True 2022-11-27T16:03:08.960 INFO:teuthology.orchestra.run.smithi170.stdout: mds op complaint time = 180 2022-11-27T16:03:08.961 INFO:teuthology.orchestra.run.smithi170.stdout: mds verify scatter = True 2022-11-27T16:03:08.961 INFO:teuthology.orchestra.run.smithi170.stdout: osd op complaint time = 180 2022-11-27T16:03:08.961 INFO:teuthology.orchestra.run.smithi170.stdout: rados mon op timeout = 900 2022-11-27T16:03:08.961 INFO:teuthology.orchestra.run.smithi170.stdout: rados osd op timeout = 900 2022-11-27T16:03:08.962 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T16:03:08.962 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T16:03:09.020 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T16:03:09.021 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:09.021 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T16:03:09.040 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:09.040 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T16:03:09.097 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi105.front.sepia.ceph.com 2022-11-27T16:03:09.098 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:09.098 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:03:09.099 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:03:09.174 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:09.175 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:09.233 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi170.front.sepia.ceph.com 2022-11-27T16:03:09.233 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:09.234 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:03:09.234 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:03:09.271 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:09.271 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:09.329 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T16:03:09.329 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T16:03:09.330 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:03:09.382 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:03:09.385 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:03:09.452 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:03:09.455 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T16:03:09.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:03:09.510 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:03:09.513 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T16:03:09.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:03:09.633 INFO:teuthology.orchestra.run.smithi105.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:03:09.636 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T16:03:09.663 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:03:09.717 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:03:09.720 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T16:03:09.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:03:09.841 INFO:teuthology.orchestra.run.smithi170.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:03:09.843 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-11-27T16:03:09.870 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T16:03:09.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T16:03:09.915 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T16:03:09.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-11-27T16:03:09.971 INFO:teuthology.orchestra.run.smithi105.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-11-27T16:03:09.983 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T16:03:10.027 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T16:03:10.040 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-11-27T16:03:10.123 INFO:teuthology.orchestra.run.smithi170.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-11-27T16:03:10.135 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T16:03:10.136 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi105.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi170.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:03:10.136 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T16:03:10.163 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:10.163 INFO:tasks.ceph:role: osd.0 2022-11-27T16:03:10.164 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com 2022-11-27T16:03:10.164 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:03:10.238 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:10.239 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:10.240 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:10.241 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:10.243 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:10.243 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:10.243 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:10.244 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:10.659 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2022-11-27T16:03:10.661 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2022-11-27T16:03:10.661 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T16:03:10.849 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T16:03:10.883 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T16:03:10.951 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:10.951 INFO:tasks.ceph:role: osd.1 2022-11-27T16:03:10.951 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com 2022-11-27T16:03:10.952 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:03:11.025 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:11.025 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:11.026 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:11.026 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:11.027 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:11.027 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:11.028 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:11.028 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:11.028 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:11.028 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:11.449 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2022-11-27T16:03:11.451 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2022-11-27T16:03:11.452 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T16:03:11.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T16:03:11.591 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T16:03:11.660 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:11.661 INFO:tasks.ceph:role: osd.2 2022-11-27T16:03:11.661 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com 2022-11-27T16:03:11.662 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:03:11.734 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:11.734 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:11.734 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:11.735 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:11.735 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:11.735 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:11.735 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:11.736 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:11.736 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:11.736 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:12.196 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2022-11-27T16:03:12.198 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2022-11-27T16:03:12.198 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T16:03:12.377 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T16:03:12.410 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T16:03:12.479 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:12.479 INFO:tasks.ceph:role: osd.3 2022-11-27T16:03:12.479 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com 2022-11-27T16:03:12.480 DEBUG:teuthology.orchestra.run.smithi105:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:03:12.554 INFO:teuthology.orchestra.run.smithi105.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:12.554 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:12.555 INFO:teuthology.orchestra.run.smithi105.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:12.555 INFO:teuthology.orchestra.run.smithi105.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:12.556 INFO:teuthology.orchestra.run.smithi105.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:12.556 INFO:teuthology.orchestra.run.smithi105.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:12.556 INFO:teuthology.orchestra.run.smithi105.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:12.557 INFO:teuthology.orchestra.run.smithi105.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:12.557 INFO:teuthology.orchestra.run.smithi105.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:12.558 INFO:teuthology.orchestra.run.smithi105.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:12.987 INFO:teuthology.orchestra.run.smithi105.stdout:Discarding blocks...Done. 2022-11-27T16:03:12.988 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi105.front.sepia.ceph.com -o noatime 2022-11-27T16:03:12.989 DEBUG:teuthology.orchestra.run.smithi105:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T16:03:13.173 DEBUG:teuthology.orchestra.run.smithi105:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T16:03:13.211 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:13.298 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:13.295+0000 7f527e3a45c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T16:03:13.298 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:13.295+0000 7f527e3a45c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T16:03:13.299 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:13.295+0000 7f527e3a45c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:03:13.299 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:13.295+0000 7f527e3a45c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:03:13.299 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:13.295+0000 7f527e3a45c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:03:13.300 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:13.295+0000 7f527e3a45c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-27T16:03:16.141 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T16:03:16.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:16.266 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:16.258+0000 7fb2427885c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T16:03:16.267 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:16.258+0000 7fb2427885c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T16:03:16.267 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:16.258+0000 7fb2427885c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:03:16.267 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:16.258+0000 7fb2427885c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:03:16.267 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:16.258+0000 7fb2427885c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:03:16.268 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:16.259+0000 7fb2427885c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-27T16:03:19.110 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T16:03:19.146 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:19.234+0000 7ff308d365c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T16:03:19.238 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:19.234+0000 7ff308d365c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T16:03:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:19.234+0000 7ff308d365c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:03:19.239 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:19.234+0000 7ff308d365c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:03:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:19.234+0000 7ff308d365c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:03:19.240 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:19.235+0000 7ff308d365c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-27T16:03:22.084 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T16:03:22.112 DEBUG:teuthology.orchestra.run.smithi105:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:22.198 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:22.196+0000 7fb38ab795c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T16:03:22.199 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:22.196+0000 7fb38ab795c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T16:03:22.200 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:22.196+0000 7fb38ab795c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:03:22.200 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:22.196+0000 7fb38ab795c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:03:22.200 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:22.196+0000 7fb38ab795c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:03:22.201 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:22.197+0000 7fb38ab795c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-27T16:03:25.047 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T16:03:25.081 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-11-27T16:03:25.115 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:25.116 INFO:tasks.ceph:role: osd.4 2022-11-27T16:03:25.116 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com 2022-11-27T16:03:25.117 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:03:25.186 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:25.187 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:25.187 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:25.187 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:25.187 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:25.188 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:25.188 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:25.188 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:25.188 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:25.189 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:25.210 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2022-11-27T16:03:25.234 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2022-11-27T16:03:25.235 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-11-27T16:03:25.317 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-11-27T16:03:25.391 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-11-27T16:03:25.459 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:25.459 INFO:tasks.ceph:role: osd.5 2022-11-27T16:03:25.460 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com 2022-11-27T16:03:25.460 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:03:25.526 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:25.526 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:25.527 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:25.527 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:25.527 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:25.527 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:25.528 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:25.528 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:25.528 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:25.528 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:25.549 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2022-11-27T16:03:25.550 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2022-11-27T16:03:25.551 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-11-27T16:03:25.631 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-11-27T16:03:25.703 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-11-27T16:03:25.770 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:25.770 INFO:tasks.ceph:role: osd.6 2022-11-27T16:03:25.771 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com 2022-11-27T16:03:25.771 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:03:25.836 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:25.837 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:25.837 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:25.838 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:25.838 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:25.838 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:25.838 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:25.839 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:25.839 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:25.839 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:25.859 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2022-11-27T16:03:25.860 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2022-11-27T16:03:25.861 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-11-27T16:03:25.945 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-11-27T16:03:26.015 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-11-27T16:03:26.082 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:03:26.083 INFO:tasks.ceph:role: osd.7 2022-11-27T16:03:26.083 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com 2022-11-27T16:03:26.084 DEBUG:teuthology.orchestra.run.smithi170:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:03:26.149 INFO:teuthology.orchestra.run.smithi170.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:03:26.150 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:03:26.150 INFO:teuthology.orchestra.run.smithi170.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:03:26.150 INFO:teuthology.orchestra.run.smithi170.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:03:26.150 INFO:teuthology.orchestra.run.smithi170.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:03:26.151 INFO:teuthology.orchestra.run.smithi170.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:03:26.151 INFO:teuthology.orchestra.run.smithi170.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:03:26.151 INFO:teuthology.orchestra.run.smithi170.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:03:26.151 INFO:teuthology.orchestra.run.smithi170.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:03:26.152 INFO:teuthology.orchestra.run.smithi170.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:03:26.172 INFO:teuthology.orchestra.run.smithi170.stdout:Discarding blocks...Done. 2022-11-27T16:03:26.174 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi170.front.sepia.ceph.com -o noatime 2022-11-27T16:03:26.174 DEBUG:teuthology.orchestra.run.smithi170:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-11-27T16:03:26.258 DEBUG:teuthology.orchestra.run.smithi170:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-11-27T16:03:26.329 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:26.413+0000 7f9658c085c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-11-27T16:03:26.417 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:26.413+0000 7f9658c085c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-11-27T16:03:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:26.413+0000 7f9658c085c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:03:26.418 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:26.413+0000 7f9658c085c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:03:26.419 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:26.413+0000 7f9658c085c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:03:26.419 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:26.414+0000 7f9658c085c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-11-27T16:03:29.259 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-11-27T16:03:29.327 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:29.414 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:29.411+0000 7f5c1271c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-11-27T16:03:29.414 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:29.411+0000 7f5c1271c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-11-27T16:03:29.415 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:29.411+0000 7f5c1271c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:03:29.415 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:29.411+0000 7f5c1271c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:03:29.416 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:29.411+0000 7f5c1271c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:03:29.416 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:29.411+0000 7f5c1271c5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-11-27T16:03:32.255 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-11-27T16:03:32.285 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:32.373+0000 7fc9786a25c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-11-27T16:03:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:32.373+0000 7fc9786a25c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-11-27T16:03:32.376 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:32.373+0000 7fc9786a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:03:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:32.373+0000 7fc9786a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:03:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:32.373+0000 7fc9786a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:03:32.377 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:32.373+0000 7fc9786a25c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-11-27T16:03:35.211 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-11-27T16:03:35.241 DEBUG:teuthology.orchestra.run.smithi170:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:35.328 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:35.325+0000 7fe9676e25c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-11-27T16:03:35.328 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:35.325+0000 7fe9676e25c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-11-27T16:03:35.329 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:35.325+0000 7fe9676e25c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:03:35.329 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:35.325+0000 7fe9676e25c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:03:35.329 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:35.325+0000 7fe9676e25c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:03:35.329 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:35.326+0000 7fe9676e25c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-11-27T16:03:36.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:03:36.772 DEBUG:teuthology.orchestra.run.smithi170:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:03:38.170 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-11-27T16:03:38.198 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T16:03:38.199 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:38.199 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T16:03:38.233 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:38.233 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T16:03:38.303 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:38.303 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T16:03:38.371 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:38.372 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T16:03:38.440 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:38.440 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T16:03:38.508 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:38.508 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T16:03:38.576 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:38.577 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T16:03:38.643 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:38.643 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T16:03:38.671 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:38.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T16:03:38.739 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:38.740 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-11-27T16:03:38.807 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:38.807 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-11-27T16:03:38.874 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:38.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-11-27T16:03:38.940 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:38.941 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-11-27T16:03:39.008 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:39.009 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-11-27T16:03:39.074 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:39.075 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T16:03:39.094 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:39.094 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-11-27T16:03:39.152 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:39.152 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T16:03:39.170 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:39.171 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-11-27T16:03:39.228 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T16:03:39.229 DEBUG:teuthology.orchestra.run.smithi105:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:03:39.231 DEBUG:teuthology.orchestra.run.smithi170:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:03:39.273 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.y] 2022-11-27T16:03:39.274 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNj4zTCFhAAvCdY9WDy+OxLKlDvsJg8XQ== 2022-11-27T16:03:39.274 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.a] 2022-11-27T16:03:39.275 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNjjRJmHhAAdTppLNkX/rkZSmW8aMIjrw== 2022-11-27T16:03:39.275 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.c] 2022-11-27T16:03:39.275 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNjwk64JRAA60/7gMlRerSabDxDkyvg6g== 2022-11-27T16:03:39.277 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.0] 2022-11-27T16:03:39.277 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBBioNjwTSpERAA9ORcVR/uGU9Os8StohmuzQ== 2022-11-27T16:03:39.278 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.1] 2022-11-27T16:03:39.278 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBEioNjf3p7DxAAhRv7UKmAHlZd8JyuuM8w7A== 2022-11-27T16:03:39.278 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.2] 2022-11-27T16:03:39.278 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBHioNjRbQLDhAA6lAzJsoU6XF5XafImKk1Hw== 2022-11-27T16:03:39.278 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.3] 2022-11-27T16:03:39.279 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBKioNjPOjCCxAAfg2V1GAeJQUDezA3xZIlbg== 2022-11-27T16:03:39.279 INFO:teuthology.orchestra.run.smithi105.stdout:[mgr.x] 2022-11-27T16:03:39.279 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNjHYvuGhAAKgI6JYq3l0nutbyj+VkCyw== 2022-11-27T16:03:39.279 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.b] 2022-11-27T16:03:39.280 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNjMAW7KhAAHKR8UU6d39vkHNV6M878yQ== 2022-11-27T16:03:39.280 INFO:teuthology.orchestra.run.smithi105.stdout:[mds.d] 2022-11-27T16:03:39.280 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNjcm4WMhAA/+ba2rYJUwSiu8i6TWpW5w== 2022-11-27T16:03:39.280 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.4] 2022-11-27T16:03:39.281 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBOioNjgNC5GBAAn4bycD5wuKkI5lTixN+Sxw== 2022-11-27T16:03:39.281 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.5] 2022-11-27T16:03:39.281 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBRioNj6QyXGBAAW/QlpoDP19EgzxPFAlx3DA== 2022-11-27T16:03:39.281 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.6] 2022-11-27T16:03:39.281 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBUioNjqQRTFhAAtJOcxGGiwwple3PKkvv1mw== 2022-11-27T16:03:39.282 INFO:teuthology.orchestra.run.smithi105.stdout:[osd.7] 2022-11-27T16:03:39.282 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQBXioNjo797ExAAjSFDObqw+/yW3GobxILJ8Q== 2022-11-27T16:03:39.282 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2022-11-27T16:03:39.282 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNj4J+ENhAAtGz9P5x2/cliNZaT/IghWw== 2022-11-27T16:03:39.283 INFO:teuthology.orchestra.run.smithi105.stdout:[client.2] 2022-11-27T16:03:39.283 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA9ioNjlfvaORAAizW+BI41USaRH22AHyxq3Q== 2022-11-27T16:03:39.283 INFO:teuthology.orchestra.run.smithi105.stdout:[client.1] 2022-11-27T16:03:39.283 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA+ioNjDkySARAAtNQuYhPrlxNbT/vSfsAJRQ== 2022-11-27T16:03:39.284 INFO:teuthology.orchestra.run.smithi105.stdout:[client.3] 2022-11-27T16:03:39.284 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQA+ioNjxJBIBxAASb8WyxAJcekKlNOQ46s2mQ== 2022-11-27T16:03:39.294 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.y] 2022-11-27T16:03:39.294 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNj4zTCFhAAvCdY9WDy+OxLKlDvsJg8XQ== 2022-11-27T16:03:39.295 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.a] 2022-11-27T16:03:39.295 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNjjRJmHhAAdTppLNkX/rkZSmW8aMIjrw== 2022-11-27T16:03:39.295 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.c] 2022-11-27T16:03:39.295 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNjwk64JRAA60/7gMlRerSabDxDkyvg6g== 2022-11-27T16:03:39.296 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.0] 2022-11-27T16:03:39.296 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBBioNjwTSpERAA9ORcVR/uGU9Os8StohmuzQ== 2022-11-27T16:03:39.296 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.1] 2022-11-27T16:03:39.296 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBEioNjf3p7DxAAhRv7UKmAHlZd8JyuuM8w7A== 2022-11-27T16:03:39.296 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.2] 2022-11-27T16:03:39.297 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBHioNjRbQLDhAA6lAzJsoU6XF5XafImKk1Hw== 2022-11-27T16:03:39.297 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.3] 2022-11-27T16:03:39.297 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBKioNjPOjCCxAAfg2V1GAeJQUDezA3xZIlbg== 2022-11-27T16:03:39.297 INFO:teuthology.orchestra.run.smithi170.stdout:[mgr.x] 2022-11-27T16:03:39.298 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNjHYvuGhAAKgI6JYq3l0nutbyj+VkCyw== 2022-11-27T16:03:39.298 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.b] 2022-11-27T16:03:39.298 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNjMAW7KhAAHKR8UU6d39vkHNV6M878yQ== 2022-11-27T16:03:39.298 INFO:teuthology.orchestra.run.smithi170.stdout:[mds.d] 2022-11-27T16:03:39.299 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNjcm4WMhAA/+ba2rYJUwSiu8i6TWpW5w== 2022-11-27T16:03:39.299 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.4] 2022-11-27T16:03:39.299 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBOioNjgNC5GBAAn4bycD5wuKkI5lTixN+Sxw== 2022-11-27T16:03:39.299 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.5] 2022-11-27T16:03:39.299 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBRioNj6QyXGBAAW/QlpoDP19EgzxPFAlx3DA== 2022-11-27T16:03:39.300 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.6] 2022-11-27T16:03:39.300 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBUioNjqQRTFhAAtJOcxGGiwwple3PKkvv1mw== 2022-11-27T16:03:39.300 INFO:teuthology.orchestra.run.smithi170.stdout:[osd.7] 2022-11-27T16:03:39.300 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQBXioNjo797ExAAjSFDObqw+/yW3GobxILJ8Q== 2022-11-27T16:03:39.301 INFO:teuthology.orchestra.run.smithi170.stdout:[client.0] 2022-11-27T16:03:39.301 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNj4J+ENhAAtGz9P5x2/cliNZaT/IghWw== 2022-11-27T16:03:39.301 INFO:teuthology.orchestra.run.smithi170.stdout:[client.2] 2022-11-27T16:03:39.301 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA9ioNjlfvaORAAizW+BI41USaRH22AHyxq3Q== 2022-11-27T16:03:39.301 INFO:teuthology.orchestra.run.smithi170.stdout:[client.1] 2022-11-27T16:03:39.302 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA+ioNjDkySARAAtNQuYhPrlxNbT/vSfsAJRQ== 2022-11-27T16:03:39.302 INFO:teuthology.orchestra.run.smithi170.stdout:[client.3] 2022-11-27T16:03:39.302 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQA+ioNjxJBIBxAASb8WyxAJcekKlNOQ46s2mQ== 2022-11-27T16:03:39.303 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:03:39.328 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:03:39.397 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.400 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.455 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.458 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.515 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.518 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.571 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.574 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.627 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.683 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.740 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:03:39.743 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:03:39.796 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.859 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:03:39.916 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:39.975 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:40.028 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:40.031 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:40.084 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:40.087 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:03:40.140 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.143 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.201 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.204 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.264 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.320 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.323 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:03:40.379 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T16:03:40.379 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T16:03:40.406 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:03:40.504 DEBUG:teuthology.orchestra.run.smithi105:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T16:03:40.532 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T16:03:40.561 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:03:40.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T16:03:40.686 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T16:03:40.753 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:03:40.851 DEBUG:teuthology.orchestra.run.smithi170:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T16:03:40.880 DEBUG:teuthology.orchestra.run.smithi105:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:40.883 DEBUG:teuthology.orchestra.run.smithi170:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:03:40.938 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T16:03:40.938 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T16:03:40.939 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T16:03:40.942 INFO:tasks.ceph.mon.a:Started 2022-11-27T16:03:40.942 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T16:03:40.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T16:03:40.980 INFO:tasks.ceph.mon.b:Started 2022-11-27T16:03:40.980 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T16:03:40.981 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T16:03:40.983 INFO:tasks.ceph.mon.c:Started 2022-11-27T16:03:40.983 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T16:03:40.984 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T16:03:40.984 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T16:03:40.986 INFO:tasks.ceph.mgr.y:Started 2022-11-27T16:03:40.987 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T16:03:40.987 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T16:03:40.989 INFO:tasks.ceph.mgr.x:Started 2022-11-27T16:03:40.990 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T16:03:40.990 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T16:03:41.111 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.109+0000 7fe6090c2700 1 Processor -- start 2022-11-27T16:03:41.112 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.109+0000 7fe6090c2700 1 -- start start 2022-11-27T16:03:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe6090c2700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe6090c2700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60406c130 con 0x7fe604070920 2022-11-27T16:03:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe6090c2700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60406c410 con 0x7fe6041599c0 2022-11-27T16:03:41.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe6090c2700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60406c6f0 con 0x7fe604072670 2022-11-27T16:03:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42328/0 (socket says 172.21.15.105:42328) 2022-11-27T16:03:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60359e700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe604070920 0x7fe604069390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:37010/0 (socket says 172.21.15.105:37010) 2022-11-27T16:03:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 learned_addr learned my addr 172.21.15.105:0/211302458 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:41.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:41.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.110+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:03:41.314 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.311+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:41.314 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.311+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:41.315 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.311+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:41.315 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.311+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:41.316 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.311+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-11-27T16:03:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.712+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:41.714 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.712+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.712+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:41.715 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.712+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:41.716 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:41.712+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-11-27T16:03:42.515 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:42.513+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:42.516 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:42.513+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:42.516 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:42.513+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:42.516 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:42.513+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:42.517 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:42.513+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-11-27T16:03:44.113 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 msgr2=0x7fe6040703e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:03:44.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe604072670 0x7fe6040703e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:44.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 >> v1:172.21.15.170:6789/0 conn(0x7fe6041599c0 legacy=0x7fe604071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:44.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 >> v1:172.21.15.105:6789/0 conn(0x7fe604070920 legacy=0x7fe604069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:44.114 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:44.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:44.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60406c6f0 con 0x7fe5e4003770 2022-11-27T16:03:44.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e4008dd0 con 0x7fe5e4006200 2022-11-27T16:03:44.115 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:44.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:44.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:44.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:44.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:03:44.117 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:03:44.118 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:44.118 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:44.119 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.111+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:03:44.315 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:44.316 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:44.316 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:44.316 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:44.317 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:44.317 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:03:44.318 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-11-27T16:03:44.318 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:44.319 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:44.319 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.312+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-11-27T16:03:44.715 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:44.716 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:44.716 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:03:44.716 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:44.717 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:44.717 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-11-27T16:03:44.717 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:44.717 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:44.718 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:44.718 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:44.713+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2022-11-27T16:03:45.516 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:45.517 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:45.517 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:03:45.517 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe60259c700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe60259c700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-11-27T16:03:45.518 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:03:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe602d9d700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:03:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:03:45.519 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:45.514+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2022-11-27T16:03:46.511 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:46.509+0000 7fad4d8f2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:03:46.515 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:46.509+0000 7f90de590200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:03:46.688 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:46.685+0000 7fad4d8f2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:03:46.688 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:46.685+0000 7f90de590200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:03:46.760 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:46.757+0000 7f90de590200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:03:46.760 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:46.758+0000 7fad4d8f2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:03:46.827 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:46.824+0000 7fad4d8f2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:03:46.828 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:46.825+0000 7f90de590200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:03:47.116 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.110+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 msgr2=0x7fe5e40086b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:03:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.110+0000 7fe600d99700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4006200 0x7fe5e40086b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.110+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 msgr2=0x7fe5e4005be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:03:47.117 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe600d99700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e4003770 0x7fe5e4005be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe60359e700 1 -- 172.21.15.105:0/211302458 reap_dead start 2022-11-27T16:03:47.118 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe600d99700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e40091d0 0x7fe5e400b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe600d99700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe5e4007dc0 0x7fe5e4008260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe60406c9d0 con 0x7fe5e4003c00 2022-11-27T16:03:47.119 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe60406c6f0 con 0x7fe5e4007dc0 2022-11-27T16:03:47.120 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe600d99700 1 -- 172.21.15.105:0/211302458 --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5e4008dd0 con 0x7fe5e40091d0 2022-11-27T16:03:47.120 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe60359e700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.105:3300/0 conn(0x7fe5e4007dc0 0x7fe5e4008260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e40091d0 0x7fe5e400b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe60359e700 1 -- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e40091d0 msgr2=0x7fe5e400b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:47.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe60359e700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e40091d0 0x7fe5e400b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.122 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe60359e700 1 -- 172.21.15.105:0/211302458 >> v1:172.21.15.170:6789/0 conn(0x7fe5e4003c00 legacy=0x7fe5e40066a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:47.122 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe60359e700 1 -- 172.21.15.105:0/211302458 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e4008a00 con 0x7fe5e4007dc0 2022-11-27T16:03:47.122 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe602d9d700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e40091d0 0x7fe5e400b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:03:47.123 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe60359e700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4007dc0 0x7fe5e4008260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6040cebf0 tx=0x7fe5fc00c0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a2f7101189f703 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:47.123 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe601d9b700 1 -- 172.21.15.105:0/211302458 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc00cbf0 con 0x7fe5e4007dc0 2022-11-27T16:03:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe601d9b700 1 -- 172.21.15.105:0/211302458 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc00cd50 con 0x7fe5e4007dc0 2022-11-27T16:03:47.124 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.111+0000 7fe601d9b700 1 -- 172.21.15.105:0/211302458 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc007310 con 0x7fe5e4007dc0 2022-11-27T16:03:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 -- 172.21.15.105:0/211302458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4007dc0 msgr2=0x7fe5e4008260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4007dc0 0x7fe5e4008260 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe6040cebf0 tx=0x7fe5fc00c0e0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.125 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 -- 172.21.15.105:0/211302458 shutdown_connections 2022-11-27T16:03:47.126 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 --2- 172.21.15.105:0/211302458 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4007dc0 0x7fe5e4008260 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.126 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 --2- 172.21.15.105:0/211302458 >> v2:172.21.15.170:3300/0 conn(0x7fe5e40091d0 0x7fe5e400b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.126 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 -- 172.21.15.105:0/211302458 >> 172.21.15.105:0/211302458 conn(0x7fe6040cb7e0 msgr2=0x7fe6040cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:47.127 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 -- 172.21.15.105:0/211302458 shutdown_connections 2022-11-27T16:03:47.127 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.112+0000 7fe6090c2700 1 -- 172.21.15.105:0/211302458 wait complete. 2022-11-27T16:03:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.114+0000 7fe6090c2700 1 Processor -- start 2022-11-27T16:03:47.128 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.114+0000 7fe6090c2700 1 -- start start 2022-11-27T16:03:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe6090c2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4003c00 0x7fe6041f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe6090c2700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4007dc0 0x7fe6040cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.131 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe6090c2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5e40091d0 0x7fe6040cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe6090c2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60415a570 con 0x7fe5e4003c00 2022-11-27T16:03:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe6090c2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe604159f40 con 0x7fe5e40091d0 2022-11-27T16:03:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe6090c2700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6041f2100 con 0x7fe5e4007dc0 2022-11-27T16:03:47.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe602d9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4003c00 0x7fe6041f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe602d9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4003c00 0x7fe6041f1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42410/0 (socket says 172.21.15.105:42410) 2022-11-27T16:03:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.115+0000 7fe602d9d700 1 -- 172.21.15.105:0/2736524841 learned_addr learned my addr 172.21.15.105:0/2736524841 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe60359e700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5e40091d0 0x7fe6040cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe60259c700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4007dc0 0x7fe6040cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe602d9d700 1 -- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4007dc0 msgr2=0x7fe6040cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe602d9d700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4007dc0 0x7fe6040cec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe602d9d700 1 -- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5e40091d0 msgr2=0x7fe6040cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe602d9d700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5e40091d0 0x7fe6040cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe602d9d700 1 -- 172.21.15.105:0/2736524841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe60406c130 con 0x7fe5e4003c00 2022-11-27T16:03:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe602d9d700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4003c00 0x7fe6041f1bc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8003fc0 tx=0x7fe5f8001570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe5eb7fe700 1 -- 172.21.15.105:0/2736524841 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8017460 con 0x7fe5e4003c00 2022-11-27T16:03:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe5eb7fe700 1 -- 172.21.15.105:0/2736524841 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f80175c0 con 0x7fe5e4003c00 2022-11-27T16:03:47.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe5eb7fe700 1 -- 172.21.15.105:0/2736524841 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8017970 con 0x7fe5e4003c00 2022-11-27T16:03:47.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6040c6c40 con 0x7fe5e4003c00 2022-11-27T16:03:47.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60415a190 con 0x7fe5e4003c00 2022-11-27T16:03:47.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe5eb7fe700 1 -- 172.21.15.105:0/2736524841 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe5f8027070 con 0x7fe5e4003c00 2022-11-27T16:03:47.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.116+0000 7fe5eb7fe700 1 -- 172.21.15.105:0/2736524841 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fe5f8020950 con 0x7fe5e4003c00 2022-11-27T16:03:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.118+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe604067420 con 0x7fe5e4003c00 2022-11-27T16:03:47.137 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.120+0000 7fe5eb7fe700 1 -- 172.21.15.105:0/2736524841 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fe5f803f6d0 con 0x7fe5e4003c00 2022-11-27T16:03:47.224 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.222+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe6040cc430 con 0x7fe5e4003c00 2022-11-27T16:03:47.227 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.225+0000 7fe5eb7fe700 1 -- 172.21.15.105:0/2736524841 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe5f802e020 con 0x7fe5e4003c00 2022-11-27T16:03:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.226+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4003c00 msgr2=0x7fe6041f1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.226+0000 7fe6090c2700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4003c00 0x7fe6041f1bc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8003fc0 tx=0x7fe5f8001570 comp rx=0 tx=0).stop 2022-11-27T16:03:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.227+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 shutdown_connections 2022-11-27T16:03:47.229 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.227+0000 7fe6090c2700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5e40091d0 0x7fe6040cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.227+0000 7fe6090c2700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe5e4003c00 0x7fe6041f1bc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.227+0000 7fe6090c2700 1 --2- 172.21.15.105:0/2736524841 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe5e4007dc0 0x7fe6040cec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.227+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 >> 172.21.15.105:0/2736524841 conn(0x7fe6040cb7e0 msgr2=0x7fe604071f60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.227+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 shutdown_connections 2022-11-27T16:03:47.230 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:47.227+0000 7fe6090c2700 1 -- 172.21.15.105:0/2736524841 wait complete. 2022-11-27T16:03:47.236 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2022-11-27T16:03:47.246 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T16:03:47.246 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:47.247 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T16:03:47.277 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:47.277 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T16:03:47.345 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:47.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T16:03:47.375 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:47.373+0000 7f90de590200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:03:47.384 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:47.382+0000 7fad4d8f2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:03:47.411 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2022-11-27T16:03:47.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T16:03:47.478 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:47.479 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-11-27T16:03:47.484 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:47.482+0000 7f90de590200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:03:47.501 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:47.499+0000 7fad4d8f2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:03:47.509 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:47.510 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-11-27T16:03:47.565 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:47.563+0000 7f90de590200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:03:47.575 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:47.573+0000 7fad4d8f2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:03:47.577 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:47.578 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-11-27T16:03:47.644 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2022-11-27T16:03:47.645 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-11-27T16:03:47.699 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:47.697+0000 7f90de590200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:03:47.701 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:47.699+0000 7fad4d8f2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:03:47.712 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new a877ed0a-6953-4642-9d56-f1eae305b6a3 0 2022-11-27T16:03:47.797 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:47.794+0000 7fad4d8f2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:03:47.800 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:47.798+0000 7f90de590200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:03:47.862 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:47.860+0000 7fad4d8f2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:03:47.868 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:47.866+0000 7f90de590200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:03:47.873 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.871+0000 7fcf791f6700 1 Processor -- start 2022-11-27T16:03:47.873 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.871+0000 7fcf791f6700 1 -- start start 2022-11-27T16:03:47.876 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.871+0000 7fcf791f6700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcf740ad990 0x7fcf7415be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.871+0000 7fcf791f6700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.871+0000 7fcf791f6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7415f820 con 0x7fcf740ce420 2022-11-27T16:03:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.871+0000 7fcf791f6700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740cdc70 con 0x7fcf740ad990 2022-11-27T16:03:47.877 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.871+0000 7fcf791f6700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740add60 con 0x7fcf740cd2b0 2022-11-27T16:03:47.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf7359e700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf72d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcf740ad990 0x7fcf7415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf72d9d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fcf740ad990 0x7fcf7415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46012/0 (socket says 172.21.15.170:46012) 2022-11-27T16:03:47.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf7359e700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47242/0 (socket says 172.21.15.170:47242) 2022-11-27T16:03:47.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf72d9d700 1 -- 172.21.15.170:0/1854488227 learned_addr learned my addr 172.21.15.170:0/1854488227 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:47.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf7359e700 1 -- 172.21.15.170:0/1854488227 >> v2:172.21.15.170:3300/0 conn(0x7fcf740ad990 msgr2=0x7fcf7415be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:47.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf7359e700 1 --2- 172.21.15.170:0/1854488227 >> v2:172.21.15.170:3300/0 conn(0x7fcf740ad990 0x7fcf7415be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf7359e700 1 -- 172.21.15.170:0/1854488227 >> v1:172.21.15.105:6789/0 conn(0x7fcf740ce420 legacy=0x7fcf7415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf7359e700 1 -- 172.21.15.170:0/1854488227 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7415f4d0 con 0x7fcf740cd2b0 2022-11-27T16:03:47.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf71d9b700 1 -- 172.21.15.170:0/1854488227 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397644668 0 0) 0x7fcf7415f820 con 0x7fcf740ce420 2022-11-27T16:03:47.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf72d9d700 1 --2- 172.21.15.170:0/1854488227 >> v2:172.21.15.170:3300/0 conn(0x7fcf740ad990 0x7fcf7415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:47.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf7359e700 1 --2- 172.21.15.170:0/1854488227 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cd730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009dc0 tx=0x7fcf6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c3770e246f9f61 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:47.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf71d9b700 1 -- 172.21.15.170:0/1854488227 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00e070 con 0x7fcf740cd2b0 2022-11-27T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf71d9b700 1 -- 172.21.15.170:0/1854488227 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00bae0 con 0x7fcf740cd2b0 2022-11-27T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.872+0000 7fcf71d9b700 1 -- 172.21.15.170:0/1854488227 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00be90 con 0x7fcf740cd2b0 2022-11-27T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 -- 172.21.15.170:0/1854488227 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 msgr2=0x7fcf740cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 --2- 172.21.15.170:0/1854488227 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cd730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c009dc0 tx=0x7fcf6c00b330 comp rx=0 tx=0).stop 2022-11-27T16:03:47.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 -- 172.21.15.170:0/1854488227 shutdown_connections 2022-11-27T16:03:47.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 --2- 172.21.15.170:0/1854488227 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cd730 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 --2- 172.21.15.170:0/1854488227 >> v2:172.21.15.170:3300/0 conn(0x7fcf740ad990 0x7fcf7415be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 -- 172.21.15.170:0/1854488227 >> 172.21.15.170:0/1854488227 conn(0x7fcf74070a30 msgr2=0x7fcf74070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:47.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 -- 172.21.15.170:0/1854488227 shutdown_connections 2022-11-27T16:03:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.873+0000 7fcf791f6700 1 -- 172.21.15.170:0/1854488227 wait complete. 2022-11-27T16:03:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.874+0000 7fcf791f6700 1 Processor -- start 2022-11-27T16:03:47.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.874+0000 7fcf791f6700 1 -- start start 2022-11-27T16:03:47.886 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf791f6700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf740ad990 0x7fcf741e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.886 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf791f6700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf791f6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf740ce420 0x7fcf7415e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:47.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf791f6700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74147ae0 con 0x7fcf740ce420 2022-11-27T16:03:47.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf791f6700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740cc8f0 con 0x7fcf740ad990 2022-11-27T16:03:47.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf791f6700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74156f00 con 0x7fcf740cd2b0 2022-11-27T16:03:47.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf72d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf740ad990 0x7fcf741e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf7259c700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf72d9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf740ad990 0x7fcf741e71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46020/0 (socket says 172.21.15.170:46020) 2022-11-27T16:03:47.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf72d9d700 1 -- 172.21.15.170:0/5186069 learned_addr learned my addr 172.21.15.170:0/5186069 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:47.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.875+0000 7fcf7359e700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf740ce420 0x7fcf7415e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:47.889 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf72d9d700 1 -- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 msgr2=0x7fcf740cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.889 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf72d9d700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.889 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf72d9d700 1 -- 172.21.15.170:0/5186069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf740ce420 msgr2=0x7fcf7415e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:47.889 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf72d9d700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf740ce420 0x7fcf7415e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf72d9d700 1 -- 172.21.15.170:0/5186069 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64009500 con 0x7fcf740ad990 2022-11-27T16:03:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf7259c700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf72d9d700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf740ad990 0x7fcf741e71a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400ebe0 tx=0x7fcf6400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:47.890 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf5b7fe700 1 -- 172.21.15.170:0/5186069 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf64005a20 con 0x7fcf740ad990 2022-11-27T16:03:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf5b7fe700 1 -- 172.21.15.170:0/5186069 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf64005b80 con 0x7fcf740ad990 2022-11-27T16:03:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6c009a70 con 0x7fcf740ad990 2022-11-27T16:03:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf5b7fe700 1 -- 172.21.15.170:0/5186069 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400c850 con 0x7fcf740ad990 2022-11-27T16:03:47.891 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740cc330 con 0x7fcf740ad990 2022-11-27T16:03:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf5b7fe700 1 -- 172.21.15.170:0/5186069 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fcf64022070 con 0x7fcf740ad990 2022-11-27T16:03:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.876+0000 7fcf5b7fe700 1 -- 172.21.15.170:0/5186069 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcf64013470 con 0x7fcf740ad990 2022-11-27T16:03:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.877+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf740723d0 con 0x7fcf740ad990 2022-11-27T16:03:47.892 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.879+0000 7fcf5b7fe700 1 -- 172.21.15.170:0/5186069 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fcf64020060 con 0x7fcf740ad990 2022-11-27T16:03:47.992 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.990+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a877ed0a-6953-4642-9d56-f1eae305b6a3", "id": 0} v 0) v1 -- 0x7fcf7406ee40 con 0x7fcf740ad990 2022-11-27T16:03:47.998 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.996+0000 7fcf5b7fe700 1 -- 172.21.15.170:0/5186069 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a877ed0a-6953-4642-9d56-f1eae305b6a3", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcf6400ce80 con 0x7fcf740ad990 2022-11-27T16:03:47.998 INFO:teuthology.orchestra.run.smithi170.stdout:0 2022-11-27T16:03:48.000 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf740ad990 msgr2=0x7fcf741e71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.001 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf740ad990 0x7fcf741e71a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400ebe0 tx=0x7fcf6400eef0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.001 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 shutdown_connections 2022-11-27T16:03:48.002 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcf740ce420 0x7fcf7415e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.002 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf740ad990 0x7fcf741e71a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.002 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 --2- 172.21.15.170:0/5186069 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fcf740cd2b0 0x7fcf740cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.003 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 >> 172.21.15.170:0/5186069 conn(0x7fcf74070a30 msgr2=0x7fcf7415a850 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.003 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 shutdown_connections 2022-11-27T16:03:48.004 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:47.997+0000 7fcf791f6700 1 -- 172.21.15.170:0/5186069 wait complete. 2022-11-27T16:03:48.011 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new dae5370d-59b3-458c-9821-f88f47769ab8 1 2022-11-27T16:03:48.135 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.128+0000 7f6e9dcd8700 1 Processor -- start 2022-11-27T16:03:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.128+0000 7f6e9dcd8700 1 -- start start 2022-11-27T16:03:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e9dcd8700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e9806c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e9dcd8700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f6e980cda40 0x7f6e980cdec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.136 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e9dcd8700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9806b6a0 con 0x7f6e98157fc0 2022-11-27T16:03:48.137 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e9dcd8700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e980cc500 con 0x7f6e980cda40 2022-11-27T16:03:48.137 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e9dcd8700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9806c780 con 0x7f6e980cc090 2022-11-27T16:03:48.137 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e96ffd700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e9806c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.137 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e977fe700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6e98157fc0 0x7f6e9806bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59192/0 (socket says 172.21.15.170:59192) 2022-11-27T16:03:48.138 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e977fe700 1 -- 172.21.15.170:0/1811977172 learned_addr learned my addr 172.21.15.170:0/1811977172 (peer_addr_for_me v1:172.21.15.170:0/0) 2022-11-27T16:03:48.138 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e97fff700 1 --2- 172.21.15.170:0/1811977172 >> v2:172.21.15.105:3300/0 conn(0x7f6e980cda40 0x7f6e980cdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.138 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e967fc700 1 -- 172.21.15.170:0/1811977172 <== mon.1 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319014520 0 0) 0x7f6e9806b6a0 con 0x7f6e98157fc0 2022-11-27T16:03:48.138 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e967fc700 1 -- 172.21.15.170:0/1811977172 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e780035b0 con 0x7f6e98157fc0 2022-11-27T16:03:48.139 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e96ffd700 1 -- 172.21.15.170:0/1811977172 >> v2:172.21.15.105:3300/0 conn(0x7f6e980cda40 msgr2=0x7f6e980cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:48.139 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e96ffd700 1 --2- 172.21.15.170:0/1811977172 >> v2:172.21.15.105:3300/0 conn(0x7f6e980cda40 0x7f6e980cdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.140 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e96ffd700 1 -- 172.21.15.170:0/1811977172 >> v1:172.21.15.170:6789/0 conn(0x7f6e98157fc0 legacy=0x7f6e9806bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.140 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e96ffd700 1 -- 172.21.15.170:0/1811977172 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9806b090 con 0x7f6e980cc090 2022-11-27T16:03:48.140 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.130+0000 7f6e96ffd700 1 --2- 172.21.15.170:0/1811977172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e9806c240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009840 tx=0x7f6e80009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b1cef07a4ae8dc7 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.141 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e967fc700 1 -- 172.21.15.170:0/1811977172 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011830 con 0x7f6e980cc090 2022-11-27T16:03:48.141 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e967fc700 1 -- 172.21.15.170:0/1811977172 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80011990 con 0x7f6e980cc090 2022-11-27T16:03:48.142 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e967fc700 1 -- 172.21.15.170:0/1811977172 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80011d00 con 0x7f6e980cc090 2022-11-27T16:03:48.142 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/1811977172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 msgr2=0x7f6e9806c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.143 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e9dcd8700 1 --2- 172.21.15.170:0/1811977172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e9806c240 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009840 tx=0x7f6e80009b50 comp rx=0 tx=0).stop 2022-11-27T16:03:48.143 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/1811977172 shutdown_connections 2022-11-27T16:03:48.143 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e9dcd8700 1 --2- 172.21.15.170:0/1811977172 >> v2:172.21.15.105:3300/0 conn(0x7f6e980cda40 0x7f6e980cdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.144 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e9dcd8700 1 --2- 172.21.15.170:0/1811977172 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e9806c240 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.145 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.131+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/1811977172 >> 172.21.15.170:0/1811977172 conn(0x7f6e98070a30 msgr2=0x7f6e98070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.145 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.132+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/1811977172 shutdown_connections 2022-11-27T16:03:48.145 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.132+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/1811977172 wait complete. 2022-11-27T16:03:48.146 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.133+0000 7f6e9dcd8700 1 Processor -- start 2022-11-27T16:03:48.146 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.133+0000 7f6e9dcd8700 1 -- start start 2022-11-27T16:03:48.151 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e9dcd8700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e980b2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.151 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e9dcd8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e980cda40 0x7f6e98154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.152 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e9dcd8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e98157fc0 0x7f6e98154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.152 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e9dcd8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981560e0 con 0x7f6e980cda40 2022-11-27T16:03:48.152 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e9dcd8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981554d0 con 0x7f6e98157fc0 2022-11-27T16:03:48.152 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e9dcd8700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981557b0 con 0x7f6e980cc090 2022-11-27T16:03:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e977fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e980b2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.153 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e97fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e98157fc0 0x7f6e98154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e977fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e980b2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47270/0 (socket says 172.21.15.170:47270) 2022-11-27T16:03:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e977fe700 1 -- 172.21.15.170:0/2798282686 learned_addr learned my addr 172.21.15.170:0/2798282686 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:48.154 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e96ffd700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e980cda40 0x7f6e98154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e977fe700 1 -- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e98157fc0 msgr2=0x7f6e98154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.134+0000 7f6e977fe700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e98157fc0 0x7f6e98154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.155 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e977fe700 1 -- 172.21.15.170:0/2798282686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e980cda40 msgr2=0x7f6e98154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.156 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e977fe700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e980cda40 0x7f6e98154740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.156 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e977fe700 1 -- 172.21.15.170:0/2798282686 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800094a0 con 0x7f6e980cc090 2022-11-27T16:03:48.156 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e97fff700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e98157fc0 0x7f6e98154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:48.157 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e96ffd700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e980cda40 0x7f6e98154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:03:48.157 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e977fe700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e980b2010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e90001b60 tx=0x7f6e90001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.157 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e8ffff700 1 -- 172.21.15.170:0/2798282686 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000ebb0 con 0x7f6e980cc090 2022-11-27T16:03:48.157 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e8ffff700 1 -- 172.21.15.170:0/2798282686 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9000ed30 con 0x7f6e980cc090 2022-11-27T16:03:48.158 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e98155a90 con 0x7f6e980cc090 2022-11-27T16:03:48.158 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e8ffff700 1 -- 172.21.15.170:0/2798282686 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000eea0 con 0x7f6e980cc090 2022-11-27T16:03:48.158 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e981551c0 con 0x7f6e980cc090 2022-11-27T16:03:48.158 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e8ffff700 1 -- 172.21.15.170:0/2798282686 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f6e9001d030 con 0x7f6e980cc090 2022-11-27T16:03:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.135+0000 7f6e8ffff700 1 -- 172.21.15.170:0/2798282686 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f6e900209a0 con 0x7f6e980cc090 2022-11-27T16:03:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.136+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9806d250 con 0x7f6e980cc090 2022-11-27T16:03:48.159 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.138+0000 7f6e8ffff700 1 -- 172.21.15.170:0/2798282686 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f6e9001b070 con 0x7f6e980cc090 2022-11-27T16:03:48.223 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.220+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dae5370d-59b3-458c-9821-f88f47769ab8", "id": 1} v 0) v1 -- 0x7f6e98067420 con 0x7f6e980cc090 2022-11-27T16:03:48.228 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.226+0000 7f6e8ffff700 1 -- 172.21.15.170:0/2798282686 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dae5370d-59b3-458c-9821-f88f47769ab8", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6e9000b350 con 0x7f6e980cc090 2022-11-27T16:03:48.228 INFO:teuthology.orchestra.run.smithi170.stdout:1 2022-11-27T16:03:48.229 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.226+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 msgr2=0x7f6e980b2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.230 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.226+0000 7f6e9dcd8700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e980b2010 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6e90001b60 tx=0x7f6e90001e70 comp rx=0 tx=0).stop 2022-11-27T16:03:48.230 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.227+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 shutdown_connections 2022-11-27T16:03:48.230 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.227+0000 7f6e9dcd8700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f6e980cc090 0x7f6e980b2010 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.231 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.227+0000 7f6e9dcd8700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e98157fc0 0x7f6e98154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.231 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.227+0000 7f6e9dcd8700 1 --2- 172.21.15.170:0/2798282686 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6e980cda40 0x7f6e98154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.231 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.227+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 >> 172.21.15.170:0/2798282686 conn(0x7f6e98070a30 msgr2=0x7f6e980c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.231 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.227+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 shutdown_connections 2022-11-27T16:03:48.232 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.227+0000 7f6e9dcd8700 1 -- 172.21.15.170:0/2798282686 wait complete. 2022-11-27T16:03:48.236 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:48.234+0000 7fad4d8f2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:03:48.240 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new e34db7ee-5ac2-4682-8b3d-77781aff8feb 2 2022-11-27T16:03:48.286 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:48.283+0000 7f90de590200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:03:48.363 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.360+0000 7fc6148dc700 1 Processor -- start 2022-11-27T16:03:48.363 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.360+0000 7fc6148dc700 1 -- start start 2022-11-27T16:03:48.372 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:48.367+0000 7fad4d8f2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:03:48.377 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc6148dc700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc6100ad990 0x7fc61015be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.378 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc6148dc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc6100ce420 0x7fc61015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.378 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc6148dc700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61015f820 con 0x7fc6101601f0 2022-11-27T16:03:48.379 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc6148dc700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6100add60 con 0x7fc6100ad990 2022-11-27T16:03:48.379 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc6148dc700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc61015f4d0 con 0x7fc6100ce420 2022-11-27T16:03:48.380 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc6100ce420 0x7fc61015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.380 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60dd9b700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc6100ce420 0x7fc61015fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46024/0 (socket says 172.21.15.170:46024) 2022-11-27T16:03:48.381 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60ed9d700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fc6101601f0 0x7fc6100cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59198/0 (socket says 172.21.15.170:59198) 2022-11-27T16:03:48.381 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60dd9b700 1 -- 172.21.15.170:0/3509665169 learned_addr learned my addr 172.21.15.170:0/3509665169 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:48.382 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60e59c700 1 --2- 172.21.15.170:0/3509665169 >> v2:172.21.15.105:3300/0 conn(0x7fc6100ad990 0x7fc61015be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.382 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60dd9b700 1 -- 172.21.15.170:0/3509665169 >> v2:172.21.15.105:3300/0 conn(0x7fc6100ad990 msgr2=0x7fc61015be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:48.382 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60dd9b700 1 --2- 172.21.15.170:0/3509665169 >> v2:172.21.15.105:3300/0 conn(0x7fc6100ad990 0x7fc61015be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.383 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60dd9b700 1 -- 172.21.15.170:0/3509665169 >> v1:172.21.15.170:6789/0 conn(0x7fc6101601f0 legacy=0x7fc6100cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.383 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60dd9b700 1 -- 172.21.15.170:0/3509665169 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61015c730 con 0x7fc6100ce420 2022-11-27T16:03:48.384 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.361+0000 7fc60d59a700 1 -- 172.21.15.170:0/3509665169 <== mon.1 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027684204 0 0) 0x7fc61015f820 con 0x7fc6101601f0 2022-11-27T16:03:48.384 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc60dd9b700 1 --2- 172.21.15.170:0/3509665169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6100ce420 0x7fc61015fcb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098f0 tx=0x7fc604009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dffb46005377f3e2 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.384 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc60d59a700 1 -- 172.21.15.170:0/3509665169 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604011830 con 0x7fc6100ce420 2022-11-27T16:03:48.385 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc60d59a700 1 -- 172.21.15.170:0/3509665169 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc604011990 con 0x7fc6100ce420 2022-11-27T16:03:48.386 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc60d59a700 1 -- 172.21.15.170:0/3509665169 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604011d20 con 0x7fc6100ce420 2022-11-27T16:03:48.386 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 -- 172.21.15.170:0/3509665169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6100ce420 msgr2=0x7fc61015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.388 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 --2- 172.21.15.170:0/3509665169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6100ce420 0x7fc61015fcb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc6040098f0 tx=0x7fc604009c00 comp rx=0 tx=0).stop 2022-11-27T16:03:48.388 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 -- 172.21.15.170:0/3509665169 shutdown_connections 2022-11-27T16:03:48.389 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 --2- 172.21.15.170:0/3509665169 >> v2:172.21.15.105:3300/0 conn(0x7fc6100ad990 0x7fc61015be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.389 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 --2- 172.21.15.170:0/3509665169 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6100ce420 0x7fc61015fcb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.389 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 -- 172.21.15.170:0/3509665169 >> 172.21.15.170:0/3509665169 conn(0x7fc610070a30 msgr2=0x7fc610070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.390 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 -- 172.21.15.170:0/3509665169 shutdown_connections 2022-11-27T16:03:48.390 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.362+0000 7fc6148dc700 1 -- 172.21.15.170:0/3509665169 wait complete. 2022-11-27T16:03:48.391 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 Processor -- start 2022-11-27T16:03:48.391 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 -- start start 2022-11-27T16:03:48.392 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc6100ad990 0x7fc610147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.392 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6100ce420 0x7fc6100c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.393 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6101601f0 0x7fc61014b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.393 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6100c5590 con 0x7fc6100ce420 2022-11-27T16:03:48.394 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61014bc30 con 0x7fc6101601f0 2022-11-27T16:03:48.394 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.364+0000 7fc6148dc700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc610148450 con 0x7fc6100ad990 2022-11-27T16:03:48.395 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6101601f0 0x7fc61014b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.395 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60e59c700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc6100ad990 0x7fc610147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.396 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6101601f0 0x7fc61014b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46038/0 (socket says 172.21.15.170:46038) 2022-11-27T16:03:48.396 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 -- 172.21.15.170:0/2712008833 learned_addr learned my addr 172.21.15.170:0/2712008833 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:48.397 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60dd9b700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6100ce420 0x7fc6100c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.397 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60e59c700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc6100ad990 0x7fc610147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47278/0 (socket says 172.21.15.170:47278) 2022-11-27T16:03:48.398 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 -- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc6100ad990 msgr2=0x7fc610147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.399 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc6100ad990 0x7fc610147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.399 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 -- 172.21.15.170:0/2712008833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6100ce420 msgr2=0x7fc6100c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.399 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6100ce420 0x7fc6100c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.400 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 -- 172.21.15.170:0/2712008833 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc604009550 con 0x7fc6101601f0 2022-11-27T16:03:48.400 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60e59c700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc6100ad990 0x7fc610147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:48.401 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60dd9b700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6100ce420 0x7fc6100c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:48.402 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc60ed9d700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6101601f0 0x7fc61014b6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc60800cb60 tx=0x7fc60800ce70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.402 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc5feffd700 1 -- 172.21.15.170:0/2712008833 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60800a9e0 con 0x7fc6101601f0 2022-11-27T16:03:48.403 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc5feffd700 1 -- 172.21.15.170:0/2712008833 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60800ab40 con 0x7fc6101601f0 2022-11-27T16:03:48.403 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc610148730 con 0x7fc6101601f0 2022-11-27T16:03:48.403 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc5feffd700 1 -- 172.21.15.170:0/2712008833 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6080070d0 con 0x7fc6101601f0 2022-11-27T16:03:48.404 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6101f4a80 con 0x7fc6101601f0 2022-11-27T16:03:48.404 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc5feffd700 1 -- 172.21.15.170:0/2712008833 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc60801a030 con 0x7fc6101601f0 2022-11-27T16:03:48.405 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.365+0000 7fc5feffd700 1 -- 172.21.15.170:0/2712008833 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fc60801c7d0 con 0x7fc6101601f0 2022-11-27T16:03:48.405 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.366+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6101f4c70 con 0x7fc6101601f0 2022-11-27T16:03:48.406 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.368+0000 7fc5feffd700 1 -- 172.21.15.170:0/2712008833 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fc608018070 con 0x7fc6101601f0 2022-11-27T16:03:48.426 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:48.423+0000 7f90de590200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:03:48.440 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:48.438+0000 7fad4d8f2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:03:48.452 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.450+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e34db7ee-5ac2-4682-8b3d-77781aff8feb", "id": 2} v 0) v1 -- 0x7fc610067420 con 0x7fc6101601f0 2022-11-27T16:03:48.460 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.458+0000 7fc5feffd700 1 -- 172.21.15.170:0/2712008833 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e34db7ee-5ac2-4682-8b3d-77781aff8feb", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc60801c480 con 0x7fc6101601f0 2022-11-27T16:03:48.461 INFO:teuthology.orchestra.run.smithi170.stdout:2 2022-11-27T16:03:48.462 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6101601f0 msgr2=0x7fc61014b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.462 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6101601f0 0x7fc61014b6f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc60800cb60 tx=0x7fc60800ce70 comp rx=0 tx=0).stop 2022-11-27T16:03:48.463 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 shutdown_connections 2022-11-27T16:03:48.463 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc6100ad990 0x7fc610147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.463 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc6101601f0 0x7fc61014b6f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.463 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 --2- 172.21.15.170:0/2712008833 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc6100ce420 0x7fc6100c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.464 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 >> 172.21.15.170:0/2712008833 conn(0x7fc610070a30 msgr2=0x7fc61006c510 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.464 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 shutdown_connections 2022-11-27T16:03:48.464 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.459+0000 7fc6148dc700 1 -- 172.21.15.170:0/2712008833 wait complete. 2022-11-27T16:03:48.473 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 98f2fda2-a32e-4e5c-ad39-c3d67013ccd3 3 2022-11-27T16:03:48.501 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:48.499+0000 7f90de590200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:03:48.598 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.596+0000 7ff1203b9700 1 Processor -- start 2022-11-27T16:03:48.599 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.596+0000 7ff1203b9700 1 -- start start 2022-11-27T16:03:48.614 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff1203b9700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff118158970 0x7ff11815cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.614 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff1203b9700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff11806bf90 0x7ff11806c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.614 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff1203b9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11806c8d0 con 0x7ff1180cd3b0 2022-11-27T16:03:48.615 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff1203b9700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11815d2c0 con 0x7ff11806bf90 2022-11-27T16:03:48.616 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff1203b9700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff118069510 con 0x7ff118158970 2022-11-27T16:03:48.617 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e155700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff118158970 0x7ff11815cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.617 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e155700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff118158970 0x7ff11815cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46046/0 (socket says 172.21.15.170:46046) 2022-11-27T16:03:48.617 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e155700 1 -- 172.21.15.170:0/2052165815 learned_addr learned my addr 172.21.15.170:0/2052165815 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:48.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11d954700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7ff1180cd3b0 0x7ff11806a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59210/0 (socket says 172.21.15.170:59210) 2022-11-27T16:03:48.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e956700 1 --2- 172.21.15.170:0/2052165815 >> v2:172.21.15.105:3300/0 conn(0x7ff11806bf90 0x7ff11806c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e155700 1 -- 172.21.15.170:0/2052165815 >> v2:172.21.15.105:3300/0 conn(0x7ff11806bf90 msgr2=0x7ff11806c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:48.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e155700 1 --2- 172.21.15.170:0/2052165815 >> v2:172.21.15.105:3300/0 conn(0x7ff11806bf90 0x7ff11806c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e155700 1 -- 172.21.15.170:0/2052165815 >> v1:172.21.15.170:6789/0 conn(0x7ff1180cd3b0 legacy=0x7ff11806a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11d153700 1 -- 172.21.15.170:0/2052165815 <== mon.1 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288474346 0 0) 0x7ff11806c8d0 con 0x7ff1180cd3b0 2022-11-27T16:03:48.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.597+0000 7ff11e155700 1 -- 172.21.15.170:0/2052165815 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1180697f0 con 0x7ff118158970 2022-11-27T16:03:48.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff11e155700 1 --2- 172.21.15.170:0/2052165815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff118158970 0x7ff11815cd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff11806b850 tx=0x7ff114009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d94bf282ad261cc server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff11d153700 1 -- 172.21.15.170:0/2052165815 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114011830 con 0x7ff118158970 2022-11-27T16:03:48.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff11d153700 1 -- 172.21.15.170:0/2052165815 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114011990 con 0x7ff118158970 2022-11-27T16:03:48.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff11d153700 1 -- 172.21.15.170:0/2052165815 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114011d20 con 0x7ff118158970 2022-11-27T16:03:48.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff1203b9700 1 -- 172.21.15.170:0/2052165815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff118158970 msgr2=0x7ff11815cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff1203b9700 1 --2- 172.21.15.170:0/2052165815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff118158970 0x7ff11815cd80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff11806b850 tx=0x7ff114009bd0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff1203b9700 1 -- 172.21.15.170:0/2052165815 shutdown_connections 2022-11-27T16:03:48.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff1203b9700 1 --2- 172.21.15.170:0/2052165815 >> v2:172.21.15.105:3300/0 conn(0x7ff11806bf90 0x7ff11806c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff1203b9700 1 --2- 172.21.15.170:0/2052165815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff118158970 0x7ff11815cd80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.598+0000 7ff1203b9700 1 -- 172.21.15.170:0/2052165815 >> 172.21.15.170:0/2052165815 conn(0x7ff1180cb7e0 msgr2=0x7ff1180cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.599+0000 7ff1203b9700 1 -- 172.21.15.170:0/2052165815 shutdown_connections 2022-11-27T16:03:48.622 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.599+0000 7ff1203b9700 1 -- 172.21.15.170:0/2052165815 wait complete. 2022-11-27T16:03:48.622 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 Processor -- start 2022-11-27T16:03:48.622 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 -- start start 2022-11-27T16:03:48.622 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff11806bf90 0x7ff118159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.622 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1180cd3b0 0x7ff1181ec870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.623 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff118158970 0x7ff1181f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.623 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1181596b0 con 0x7ff118158970 2022-11-27T16:03:48.623 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11815af30 con 0x7ff1180cd3b0 2022-11-27T16:03:48.623 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff1203b9700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11815b210 con 0x7ff11806bf90 2022-11-27T16:03:48.624 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff11e155700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff11806bf90 0x7ff118159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.624 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.600+0000 7ff11d954700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1180cd3b0 0x7ff1181ec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.624 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e155700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff11806bf90 0x7ff118159170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47280/0 (socket says 172.21.15.170:47280) 2022-11-27T16:03:48.624 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e155700 1 -- 172.21.15.170:0/611291390 learned_addr learned my addr 172.21.15.170:0/611291390 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:48.625 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e956700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff118158970 0x7ff1181f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.625 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e956700 1 -- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff11806bf90 msgr2=0x7ff118159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.625 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e956700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff11806bf90 0x7ff118159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.625 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e956700 1 -- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1180cd3b0 msgr2=0x7ff1181ec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.626 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e956700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1180cd3b0 0x7ff1181ec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.626 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e956700 1 -- 172.21.15.170:0/611291390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff114009550 con 0x7ff118158970 2022-11-27T16:03:48.626 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11d954700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1180cd3b0 0x7ff1181ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:48.626 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e155700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff11806bf90 0x7ff118159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:48.626 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff11e956700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff118158970 0x7ff1181f2a10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efc0 tx=0x7ff11000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.627 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff106ffd700 1 -- 172.21.15.170:0/611291390 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11000cc90 con 0x7ff118158970 2022-11-27T16:03:48.627 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff106ffd700 1 -- 172.21.15.170:0/611291390 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11000f040 con 0x7ff118158970 2022-11-27T16:03:48.627 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11815b4f0 con 0x7ff118158970 2022-11-27T16:03:48.627 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff106ffd700 1 -- 172.21.15.170:0/611291390 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1100146c0 con 0x7ff118158970 2022-11-27T16:03:48.628 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11815bab0 con 0x7ff118158970 2022-11-27T16:03:48.628 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff106ffd700 1 -- 172.21.15.170:0/611291390 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff110014820 con 0x7ff118158970 2022-11-27T16:03:48.628 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.601+0000 7ff106ffd700 1 -- 172.21.15.170:0/611291390 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7ff11001db10 con 0x7ff118158970 2022-11-27T16:03:48.628 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.602+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11815bcd0 con 0x7ff118158970 2022-11-27T16:03:48.628 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.604+0000 7ff106ffd700 1 -- 172.21.15.170:0/611291390 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7ff11003b6d0 con 0x7ff118158970 2022-11-27T16:03:48.629 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:48.600+0000 7fad4d8f2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:03:48.672 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:48.670+0000 7f90de590200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:03:48.689 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.687+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd new", "uuid": "98f2fda2-a32e-4e5c-ad39-c3d67013ccd3", "id": 3} v 0) v1 -- 0x7ff118067420 con 0x7ff118158970 2022-11-27T16:03:48.696 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.694+0000 7ff106ffd700 1 -- 172.21.15.170:0/611291390 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "98f2fda2-a32e-4e5c-ad39-c3d67013ccd3", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff11002a070 con 0x7ff118158970 2022-11-27T16:03:48.697 INFO:teuthology.orchestra.run.smithi170.stdout:3 2022-11-27T16:03:48.699 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff118158970 msgr2=0x7ff1181f2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.699 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff118158970 0x7ff1181f2a10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff11000efc0 tx=0x7ff11000c3a0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.700 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 shutdown_connections 2022-11-27T16:03:48.700 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff11806bf90 0x7ff118159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.700 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff118158970 0x7ff1181f2a10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.701 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 --2- 172.21.15.170:0/611291390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1180cd3b0 0x7ff1181ec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.701 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 >> 172.21.15.170:0/611291390 conn(0x7ff1180cb7e0 msgr2=0x7ff11806a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.701 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 shutdown_connections 2022-11-27T16:03:48.702 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.695+0000 7ff1203b9700 1 -- 172.21.15.170:0/611291390 wait complete. 2022-11-27T16:03:48.709 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new ec8582a4-93d8-4e63-8b94-5386bf561c11 4 2022-11-27T16:03:48.744 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:48.741+0000 7fad4d8f2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:03:48.820 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:48.818+0000 7f90de590200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:03:48.870 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.867+0000 7f674badf700 1 Processor -- start 2022-11-27T16:03:48.871 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.868+0000 7f674badf700 1 -- start start 2022-11-27T16:03:48.877 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.868+0000 7f674badf700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f674415be80 0x7f674406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.877 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.868+0000 7f674badf700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674415c250 con 0x7f6744071f60 2022-11-27T16:03:48.877 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.868+0000 7f674badf700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67440ce580 con 0x7f6744070f60 2022-11-27T16:03:48.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.868+0000 7f674badf700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6744070b40 con 0x7f674415be80 2022-11-27T16:03:48.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f674907a700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7f6744070f60 0x7f67440cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59214/0 (socket says 172.21.15.170:59214) 2022-11-27T16:03:48.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f674907a700 1 -- 172.21.15.170:0/1721236125 learned_addr learned my addr 172.21.15.170:0/1721236125 (peer_addr_for_me v1:172.21.15.170:0/0) 2022-11-27T16:03:48.878 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f674987b700 1 --2- 172.21.15.170:0/1721236125 >> v2:172.21.15.105:3300/0 conn(0x7f674415be80 0x7f674406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f6748879700 1 -- 172.21.15.170:0/1721236125 <== mon.1 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355392752 0 0) 0x7f6734001110 con 0x7f6744070f60 2022-11-27T16:03:48.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f6748879700 1 -- 172.21.15.170:0/1721236125 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f672c0035b0 con 0x7f6744070f60 2022-11-27T16:03:48.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f6748879700 1 -- 172.21.15.170:0/1721236125 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499083474 0 0) 0x7f673c001490 con 0x7f6744071f60 2022-11-27T16:03:48.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f6748879700 1 -- 172.21.15.170:0/1721236125 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6734001110 con 0x7f6744071f60 2022-11-27T16:03:48.879 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f674987b700 1 -- 172.21.15.170:0/1721236125 >> v1:172.21.15.170:6789/0 conn(0x7f6744070f60 legacy=0x7f67440cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f674987b700 1 -- 172.21.15.170:0/1721236125 >> v1:172.21.15.105:6789/0 conn(0x7f6744071f60 legacy=0x7f6744069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f674987b700 1 -- 172.21.15.170:0/1721236125 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440ce7c0 con 0x7f674415be80 2022-11-27T16:03:48.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f674987b700 1 --2- 172.21.15.170:0/1721236125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f674415be80 0x7f674406a9b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6740009840 tx=0x7f6740009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e220c6da234425c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.880 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f6748879700 1 -- 172.21.15.170:0/1721236125 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740011830 con 0x7f674415be80 2022-11-27T16:03:48.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f6748879700 1 -- 172.21.15.170:0/1721236125 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6740011990 con 0x7f674415be80 2022-11-27T16:03:48.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.869+0000 7f6748879700 1 -- 172.21.15.170:0/1721236125 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6740011d20 con 0x7f674415be80 2022-11-27T16:03:48.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.870+0000 7f674badf700 1 -- 172.21.15.170:0/1721236125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f674415be80 msgr2=0x7f674406a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.881 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.870+0000 7f674badf700 1 --2- 172.21.15.170:0/1721236125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f674415be80 0x7f674406a9b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6740009840 tx=0x7f6740009b50 comp rx=0 tx=0).stop 2022-11-27T16:03:48.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.870+0000 7f674badf700 1 -- 172.21.15.170:0/1721236125 shutdown_connections 2022-11-27T16:03:48.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.870+0000 7f674badf700 1 --2- 172.21.15.170:0/1721236125 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f674415be80 0x7f674406a9b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.870+0000 7f674badf700 1 -- 172.21.15.170:0/1721236125 >> 172.21.15.170:0/1721236125 conn(0x7f67440cb7e0 msgr2=0x7f67440cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.870+0000 7f674badf700 1 -- 172.21.15.170:0/1721236125 shutdown_connections 2022-11-27T16:03:48.882 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.870+0000 7f674badf700 1 -- 172.21.15.170:0/1721236125 wait complete. 2022-11-27T16:03:48.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.871+0000 7f674badf700 1 Processor -- start 2022-11-27T16:03:48.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.871+0000 7f674badf700 1 -- start start 2022-11-27T16:03:48.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674badf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744070f60 0x7f6744157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674badf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744071f60 0x7f6744157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.883 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674badf700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f674415be80 0x7f6744158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:48.884 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674badf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674415c3f0 con 0x7f6744071f60 2022-11-27T16:03:48.884 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674badf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67441577c0 con 0x7f6744070f60 2022-11-27T16:03:48.884 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674badf700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6744158730 con 0x7f674415be80 2022-11-27T16:03:48.884 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744070f60 0x7f6744157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.885 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744070f60 0x7f6744157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46066/0 (socket says 172.21.15.170:46066) 2022-11-27T16:03:48.885 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 -- 172.21.15.170:0/1106207983 learned_addr learned my addr 172.21.15.170:0/1106207983 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:48.885 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674a07c700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f674415be80 0x7f6744158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.885 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674907a700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744071f60 0x7f6744157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:48.885 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 -- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f674415be80 msgr2=0x7f6744158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.886 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f674415be80 0x7f6744158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.886 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 -- 172.21.15.170:0/1106207983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744071f60 msgr2=0x7f6744157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.886 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744071f60 0x7f6744157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.886 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.872+0000 7f674987b700 1 -- 172.21.15.170:0/1106207983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400094a0 con 0x7f6744070f60 2022-11-27T16:03:48.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f674987b700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744070f60 0x7f6744157280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67400090b0 tx=0x7f6740004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:48.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f67327fc700 1 -- 172.21.15.170:0/1106207983 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674001a6e0 con 0x7f6744070f60 2022-11-27T16:03:48.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f67327fc700 1 -- 172.21.15.170:0/1106207983 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674001a840 con 0x7f6744070f60 2022-11-27T16:03:48.887 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6744158a10 con 0x7f6744070f60 2022-11-27T16:03:48.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f67327fc700 1 -- 172.21.15.170:0/1106207983 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674001abb0 con 0x7f6744070f60 2022-11-27T16:03:48.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674415c990 con 0x7f6744070f60 2022-11-27T16:03:48.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f67327fc700 1 -- 172.21.15.170:0/1106207983 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f674002b030 con 0x7f6744070f60 2022-11-27T16:03:48.888 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.873+0000 7f67327fc700 1 -- 172.21.15.170:0/1106207983 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f674002d8b0 con 0x7f6744070f60 2022-11-27T16:03:48.889 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.874+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674415cb80 con 0x7f6744070f60 2022-11-27T16:03:48.889 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:48.880+0000 7fad4d8f2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:03:48.890 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.876+0000 7f67327fc700 1 -- 172.21.15.170:0/1106207983 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f6740029070 con 0x7f6744070f60 2022-11-27T16:03:48.961 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.958+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ec8582a4-93d8-4e63-8b94-5386bf561c11", "id": 4} v 0) v1 -- 0x7f6744067420 con 0x7f6744070f60 2022-11-27T16:03:48.967 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.965+0000 7f67327fc700 1 -- 172.21.15.170:0/1106207983 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec8582a4-93d8-4e63-8b94-5386bf561c11", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6740015020 con 0x7f6744070f60 2022-11-27T16:03:48.968 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:48.965+0000 7f90de590200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:03:48.968 INFO:teuthology.orchestra.run.smithi170.stdout:4 2022-11-27T16:03:48.969 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744070f60 msgr2=0x7f6744157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:48.969 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744070f60 0x7f6744157280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67400090b0 tx=0x7f6740004750 comp rx=0 tx=0).stop 2022-11-27T16:03:48.970 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 shutdown_connections 2022-11-27T16:03:48.970 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f674415be80 0x7f6744158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.970 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6744070f60 0x7f6744157280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.970 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 --2- 172.21.15.170:0/1106207983 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6744071f60 0x7f6744157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:48.971 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 >> 172.21.15.170:0/1106207983 conn(0x7f67440cb7e0 msgr2=0x7f67440c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:48.971 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 shutdown_connections 2022-11-27T16:03:48.971 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:48.966+0000 7f674badf700 1 -- 172.21.15.170:0/1106207983 wait complete. 2022-11-27T16:03:48.980 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 55b1b3a8-5dac-4f1e-b0d6-1871422ae097 5 2022-11-27T16:03:49.105 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.102+0000 7fdca6923700 1 Processor -- start 2022-11-27T16:03:49.105 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.102+0000 7fdca6923700 1 -- start start 2022-11-27T16:03:49.113 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.103+0000 7fdca6923700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fdca006d000 0x7fdca00b2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.114 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.103+0000 7fdca6923700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca006d480 con 0x7fdca00aef30 2022-11-27T16:03:49.114 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.103+0000 7fdca6923700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca015f970 con 0x7fdca00cbc90 2022-11-27T16:03:49.114 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.103+0000 7fdca6923700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca00b28b0 con 0x7fdca006d000 2022-11-27T16:03:49.114 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.103+0000 7fdca4ec0700 1 --1- >> v1:172.21.15.170:6789/0 conn(0x7fdca00cbc90 0x7fdca00cc0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:59224/0 (socket says 172.21.15.170:59224) 2022-11-27T16:03:49.115 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.103+0000 7fdca4ec0700 1 -- 172.21.15.170:0/1610073331 learned_addr learned my addr 172.21.15.170:0/1610073331 (peer_addr_for_me v1:172.21.15.170:0/0) 2022-11-27T16:03:49.115 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.103+0000 7fdc9f7fe700 1 --2- 172.21.15.170:0/1610073331 >> v2:172.21.15.105:3300/0 conn(0x7fdca006d000 0x7fdca00b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.115 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9effd700 1 -- 172.21.15.170:0/1610073331 <== mon.1 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316013588 0 0) 0x7fdca015f970 con 0x7fdca00cbc90 2022-11-27T16:03:49.115 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9effd700 1 -- 172.21.15.170:0/1610073331 --> v1:172.21.15.170:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc8c0035b0 con 0x7fdca00cbc90 2022-11-27T16:03:49.116 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9effd700 1 -- 172.21.15.170:0/1610073331 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 473000072 0 0) 0x7fdc90001110 con 0x7fdca00aef30 2022-11-27T16:03:49.116 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9effd700 1 -- 172.21.15.170:0/1610073331 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdca015f970 con 0x7fdca00aef30 2022-11-27T16:03:49.116 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9f7fe700 1 -- 172.21.15.170:0/1610073331 >> v1:172.21.15.170:6789/0 conn(0x7fdca00cbc90 legacy=0x7fdca00cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.116 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9f7fe700 1 -- 172.21.15.170:0/1610073331 >> v1:172.21.15.105:6789/0 conn(0x7fdca00aef30 legacy=0x7fdca0160150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.116 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9f7fe700 1 -- 172.21.15.170:0/1610073331 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca006d6c0 con 0x7fdca006d000 2022-11-27T16:03:49.117 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9f7fe700 1 --2- 172.21.15.170:0/1610073331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca006d000 0x7fdca00b2370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009840 tx=0x7fdc94009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892a292bb9438150 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.117 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9effd700 1 -- 172.21.15.170:0/1610073331 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94011830 con 0x7fdca006d000 2022-11-27T16:03:49.117 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9effd700 1 -- 172.21.15.170:0/1610073331 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc94011990 con 0x7fdca006d000 2022-11-27T16:03:49.117 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.104+0000 7fdc9effd700 1 -- 172.21.15.170:0/1610073331 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc94011d20 con 0x7fdca006d000 2022-11-27T16:03:49.118 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.105+0000 7fdca6923700 1 -- 172.21.15.170:0/1610073331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca006d000 msgr2=0x7fdca00b2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.118 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.105+0000 7fdca6923700 1 --2- 172.21.15.170:0/1610073331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca006d000 0x7fdca00b2370 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fdc94009840 tx=0x7fdc94009b50 comp rx=0 tx=0).stop 2022-11-27T16:03:49.118 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.105+0000 7fdca6923700 1 -- 172.21.15.170:0/1610073331 shutdown_connections 2022-11-27T16:03:49.118 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.105+0000 7fdca6923700 1 --2- 172.21.15.170:0/1610073331 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca006d000 0x7fdca00b2370 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.118 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.105+0000 7fdca6923700 1 -- 172.21.15.170:0/1610073331 >> 172.21.15.170:0/1610073331 conn(0x7fdca00cff30 msgr2=0x7fdca00d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.105+0000 7fdca6923700 1 -- 172.21.15.170:0/1610073331 shutdown_connections 2022-11-27T16:03:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.105+0000 7fdca6923700 1 -- 172.21.15.170:0/1610073331 wait complete. 2022-11-27T16:03:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.106+0000 7fdca6923700 1 Processor -- start 2022-11-27T16:03:49.119 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.106+0000 7fdca6923700 1 -- start start 2022-11-27T16:03:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca6923700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdca006d000 0x7fdca015dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca6923700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdca00aef30 0x7fdca00b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca6923700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca00cbc90 0x7fdca00b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca6923700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca015c630 con 0x7fdca00cbc90 2022-11-27T16:03:49.120 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca6923700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca015e300 con 0x7fdca006d000 2022-11-27T16:03:49.121 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca6923700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca015e5e0 con 0x7fdca00aef30 2022-11-27T16:03:49.121 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdca006d000 0x7fdca015dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.121 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdca00aef30 0x7fdca00b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.122 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdca006d000 0x7fdca015dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:46082/0 (socket says 172.21.15.170:46082) 2022-11-27T16:03:49.122 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9f7fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdca00aef30 0x7fdca00b4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:47300/0 (socket says 172.21.15.170:47300) 2022-11-27T16:03:49.122 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9ffff700 1 -- 172.21.15.170:0/3637419724 learned_addr learned my addr 172.21.15.170:0/3637419724 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca4ec0700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca00cbc90 0x7fdca00b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9f7fe700 1 -- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdca006d000 msgr2=0x7fdca015dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9f7fe700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdca006d000 0x7fdca015dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.123 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9f7fe700 1 -- 172.21.15.170:0/3637419724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca00cbc90 msgr2=0x7fdca00b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9f7fe700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca00cbc90 0x7fdca00b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9f7fe700 1 -- 172.21.15.170:0/3637419724 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc940094a0 con 0x7fdca00aef30 2022-11-27T16:03:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdca4ec0700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca00cbc90 0x7fdca00b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:49.124 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.107+0000 7fdc9ffff700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdca006d000 0x7fdca015dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:49.125 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdc9f7fe700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdca00aef30 0x7fdca00b4fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc94016000 tx=0x7fdc94004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.125 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdc9cff9700 1 -- 172.21.15.170:0/3637419724 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9401a6e0 con 0x7fdca00aef30 2022-11-27T16:03:49.125 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdc9cff9700 1 -- 172.21.15.170:0/3637419724 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9401a860 con 0x7fdca00aef30 2022-11-27T16:03:49.125 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca015e8c0 con 0x7fdca00aef30 2022-11-27T16:03:49.125 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdc9cff9700 1 -- 172.21.15.170:0/3637419724 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9401ac10 con 0x7fdca00aef30 2022-11-27T16:03:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca015ee80 con 0x7fdca00aef30 2022-11-27T16:03:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdc9cff9700 1 -- 172.21.15.170:0/3637419724 <== mon.2 v2:172.21.15.170:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fdc9402b030 con 0x7fdca00aef30 2022-11-27T16:03:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.108+0000 7fdc9cff9700 1 -- 172.21.15.170:0/3637419724 <== mon.2 v2:172.21.15.170:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7fdc9402d920 con 0x7fdca00aef30 2022-11-27T16:03:49.126 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.109+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca0067420 con 0x7fdca00aef30 2022-11-27T16:03:49.127 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.111+0000 7fdc9cff9700 1 -- 172.21.15.170:0/3637419724 <== mon.2 v2:172.21.15.170:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fdc94029070 con 0x7fdca00aef30 2022-11-27T16:03:49.127 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:49.116+0000 7fad4d8f2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:03:49.196 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.194+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_command({"prefix": "osd new", "uuid": "55b1b3a8-5dac-4f1e-b0d6-1871422ae097", "id": 5} v 0) v1 -- 0x7fdca00b5eb0 con 0x7fdca00aef30 2022-11-27T16:03:49.201 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.199+0000 7fdc9cff9700 1 -- 172.21.15.170:0/3637419724 <== mon.2 v2:172.21.15.170:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "55b1b3a8-5dac-4f1e-b0d6-1871422ae097", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fdc94015020 con 0x7fdca00aef30 2022-11-27T16:03:49.201 INFO:teuthology.orchestra.run.smithi170.stdout:5 2022-11-27T16:03:49.203 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdca00aef30 msgr2=0x7fdca00b4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.203 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdca00aef30 0x7fdca00b4fa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdc94016000 tx=0x7fdc94004750 comp rx=0 tx=0).stop 2022-11-27T16:03:49.204 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 shutdown_connections 2022-11-27T16:03:49.204 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdca00cbc90 0x7fdca00b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.205 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdca006d000 0x7fdca015dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.205 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 --2- 172.21.15.170:0/3637419724 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fdca00aef30 0x7fdca00b4fa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.206 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 >> 172.21.15.170:0/3637419724 conn(0x7fdca00cff30 msgr2=0x7fdca015af70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.206 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 shutdown_connections 2022-11-27T16:03:49.206 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.200+0000 7fdca6923700 1 -- 172.21.15.170:0/3637419724 wait complete. 2022-11-27T16:03:49.214 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 64ff9585-481c-4ab8-a276-dad128c5dba9 6 2022-11-27T16:03:49.218 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:49.217+0000 7f90de590200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:03:49.251 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:49.249+0000 7fad4d8f2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:03:49.345 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b592fa700 1 Processor -- start 2022-11-27T16:03:49.346 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b592fa700 1 -- start start 2022-11-27T16:03:49.346 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b592fa700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b540ad540 0x7f8b540ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b592fa700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f8b540c9220 0x7f8b540abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.347 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b592fa700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b54158c30 con 0x7f8b54158860 2022-11-27T16:03:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b592fa700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5415d080 con 0x7f8b540c9220 2022-11-27T16:03:49.348 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b592fa700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b540ab090 con 0x7f8b540ad540 2022-11-27T16:03:49.349 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.335+0000 7f8b527fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b540ad540 0x7f8b540ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.349 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b527fc700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f8b540ad540 0x7f8b540ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49336/0 (socket says 172.21.15.170:49336) 2022-11-27T16:03:49.349 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b527fc700 1 -- 172.21.15.170:0/2720174109 learned_addr learned my addr 172.21.15.170:0/2720174109 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:49.350 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b537fe700 1 --2- 172.21.15.170:0/2720174109 >> v2:172.21.15.105:3300/0 conn(0x7f8b540c9220 0x7f8b540abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.350 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b527fc700 1 -- 172.21.15.170:0/2720174109 >> v2:172.21.15.105:3300/0 conn(0x7f8b540c9220 msgr2=0x7f8b540abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:49.351 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b527fc700 1 --2- 172.21.15.170:0/2720174109 >> v2:172.21.15.105:3300/0 conn(0x7f8b540c9220 0x7f8b540abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.351 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b527fc700 1 -- 172.21.15.170:0/2720174109 >> v1:172.21.15.105:6789/0 conn(0x7f8b54158860 legacy=0x7f8b540c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.351 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b527fc700 1 -- 172.21.15.170:0/2720174109 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540ab370 con 0x7f8b540ad540 2022-11-27T16:03:49.352 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b527fc700 1 --2- 172.21.15.170:0/2720174109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b540ad9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b5406f050 tx=0x7f8b48009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9076d8a6933d473 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.352 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b51ffb700 1 -- 172.21.15.170:0/2720174109 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48011830 con 0x7f8b540ad540 2022-11-27T16:03:49.353 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b51ffb700 1 -- 172.21.15.170:0/2720174109 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48011990 con 0x7f8b540ad540 2022-11-27T16:03:49.353 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.336+0000 7f8b51ffb700 1 -- 172.21.15.170:0/2720174109 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48011d20 con 0x7f8b540ad540 2022-11-27T16:03:49.353 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 -- 172.21.15.170:0/2720174109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 msgr2=0x7f8b540ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.354 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 --2- 172.21.15.170:0/2720174109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b540ad9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8b5406f050 tx=0x7f8b48009bd0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.354 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 -- 172.21.15.170:0/2720174109 shutdown_connections 2022-11-27T16:03:49.354 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 --2- 172.21.15.170:0/2720174109 >> v2:172.21.15.105:3300/0 conn(0x7f8b540c9220 0x7f8b540abe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.354 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 --2- 172.21.15.170:0/2720174109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b540ad9b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.355 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 -- 172.21.15.170:0/2720174109 >> 172.21.15.170:0/2720174109 conn(0x7f8b54070a30 msgr2=0x7f8b54070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.355 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 -- 172.21.15.170:0/2720174109 shutdown_connections 2022-11-27T16:03:49.355 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.337+0000 7f8b592fa700 1 -- 172.21.15.170:0/2720174109 wait complete. 2022-11-27T16:03:49.355 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.338+0000 7f8b592fa700 1 Processor -- start 2022-11-27T16:03:49.356 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.338+0000 7f8b592fa700 1 -- start start 2022-11-27T16:03:49.356 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b592fa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b541f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.356 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b592fa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b540c9220 0x7f8b541f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.356 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b592fa700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b54158860 0x7f8b540b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.357 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b592fa700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540cd2a0 con 0x7f8b540c9220 2022-11-27T16:03:49.357 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b592fa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540cc690 con 0x7f8b540ad540 2022-11-27T16:03:49.357 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b592fa700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540cc970 con 0x7f8b54158860 2022-11-27T16:03:49.357 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b541f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.358 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b537fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b54158860 0x7f8b540b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.358 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b541f1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49344/0 (socket says 172.21.15.170:49344) 2022-11-27T16:03:49.358 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b537fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b54158860 0x7f8b540b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.170:40236/0 (socket says 172.21.15.170:40236) 2022-11-27T16:03:49.358 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 -- 172.21.15.170:0/3272551418 learned_addr learned my addr 172.21.15.170:0/3272551418 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:49.358 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b527fc700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b540c9220 0x7f8b541f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.359 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 -- 172.21.15.170:0/3272551418 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b54158860 msgr2=0x7f8b540b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.359 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b54158860 0x7f8b540b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.359 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 -- 172.21.15.170:0/3272551418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b540c9220 msgr2=0x7f8b541f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.359 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b540c9220 0x7f8b541f2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.360 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b52ffd700 1 -- 172.21.15.170:0/3272551418 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48009550 con 0x7f8b540ad540 2022-11-27T16:03:49.360 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.339+0000 7f8b527fc700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b540c9220 0x7f8b541f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:03:49.360 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b52ffd700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b541f1250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400c4d0 tx=0x7f8b4400c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.360 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b3b7fe700 1 -- 172.21.15.170:0/3272551418 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440052d0 con 0x7f8b540ad540 2022-11-27T16:03:49.361 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b3b7fe700 1 -- 172.21.15.170:0/3272551418 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b44005430 con 0x7f8b540ad540 2022-11-27T16:03:49.361 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b540ccc50 con 0x7f8b540ad540 2022-11-27T16:03:49.361 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b3b7fe700 1 -- 172.21.15.170:0/3272551418 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440038f0 con 0x7f8b540ad540 2022-11-27T16:03:49.361 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b540ad0c0 con 0x7f8b540ad540 2022-11-27T16:03:49.361 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b3b7fe700 1 -- 172.21.15.170:0/3272551418 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8b44019030 con 0x7f8b540ad540 2022-11-27T16:03:49.362 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.340+0000 7f8b3b7fe700 1 -- 172.21.15.170:0/3272551418 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f8b44011d10 con 0x7f8b540ad540 2022-11-27T16:03:49.362 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.341+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b540722d0 con 0x7f8b540ad540 2022-11-27T16:03:49.362 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.343+0000 7f8b3b7fe700 1 -- 172.21.15.170:0/3272551418 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f8b44017070 con 0x7f8b540ad540 2022-11-27T16:03:49.363 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:49.361+0000 7f90de590200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:03:49.427 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.425+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "64ff9585-481c-4ab8-a276-dad128c5dba9", "id": 6} v 0) v1 -- 0x7f8b54067420 con 0x7f8b540ad540 2022-11-27T16:03:49.434 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.432+0000 7f8b3b7fe700 1 -- 172.21.15.170:0/3272551418 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64ff9585-481c-4ab8-a276-dad128c5dba9", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8b54158c30 con 0x7f8b540ad540 2022-11-27T16:03:49.435 INFO:teuthology.orchestra.run.smithi170.stdout:6 2022-11-27T16:03:49.436 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 msgr2=0x7f8b541f1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.437 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b541f1250 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400c4d0 tx=0x7f8b4400c7e0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.437 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 shutdown_connections 2022-11-27T16:03:49.437 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f8b54158860 0x7f8b540b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.437 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8b540ad540 0x7f8b541f1250 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.438 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 --2- 172.21.15.170:0/3272551418 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b540c9220 0x7f8b541f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.438 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 >> 172.21.15.170:0/3272551418 conn(0x7f8b54070a30 msgr2=0x7f8b54155ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.438 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 shutdown_connections 2022-11-27T16:03:49.438 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.433+0000 7f8b592fa700 1 -- 172.21.15.170:0/3272551418 wait complete. 2022-11-27T16:03:49.447 DEBUG:teuthology.orchestra.run.smithi170:> sudo ceph --cluster ceph osd new 9a2592f7-a19d-49df-beb3-004cd10cf16b 7 2022-11-27T16:03:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.607+0000 7ff3f9838700 1 Processor -- start 2022-11-27T16:03:49.610 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.607+0000 7ff3f9838700 1 -- start start 2022-11-27T16:03:49.613 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f9838700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff3f40ad990 0x7ff3f415be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.614 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f9838700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff3f40ce420 0x7ff3f415fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.614 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f9838700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3f415f820 con 0x7ff3f41601f0 2022-11-27T16:03:49.614 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f9838700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f40add60 con 0x7ff3f40ce420 2022-11-27T16:03:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f9838700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3f415f4d0 con 0x7ff3f40ad990 2022-11-27T16:03:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff3f40ad990 0x7ff3f415be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.615 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f2ffd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff3f40ad990 0x7ff3f415be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49348/0 (socket says 172.21.15.170:49348) 2022-11-27T16:03:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f2ffd700 1 -- 172.21.15.170:0/3443609036 learned_addr learned my addr 172.21.15.170:0/3443609036 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.608+0000 7ff3f37fe700 1 --1- 172.21.15.170:0/3443609036 >> v1:172.21.15.170:6789/0 conn(0x7ff3f41601f0 0x7ff3f40cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.170:6789/0 says I am v1:172.21.15.170:43934/0 (socket says 172.21.15.170:43934) 2022-11-27T16:03:49.616 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f27fc700 1 --2- 172.21.15.170:0/3443609036 >> v2:172.21.15.105:3300/0 conn(0x7ff3f40ce420 0x7ff3f415fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f2ffd700 1 -- 172.21.15.170:0/3443609036 >> v2:172.21.15.105:3300/0 conn(0x7ff3f40ce420 msgr2=0x7ff3f415fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f2ffd700 1 --2- 172.21.15.170:0/3443609036 >> v2:172.21.15.105:3300/0 conn(0x7ff3f40ce420 0x7ff3f415fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.617 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f2ffd700 1 -- 172.21.15.170:0/3443609036 >> v1:172.21.15.170:6789/0 conn(0x7ff3f41601f0 legacy=0x7ff3f40cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f2ffd700 1 -- 172.21.15.170:0/3443609036 --> v2:172.21.15.170:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f415c730 con 0x7ff3f40ad990 2022-11-27T16:03:49.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f1ffb700 1 -- 172.21.15.170:0/3443609036 <== mon.1 v1:172.21.15.170:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258573887 0 0) 0x7ff3f415f820 con 0x7ff3f41601f0 2022-11-27T16:03:49.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f27fc700 1 --2- 172.21.15.170:0/3443609036 >> v2:172.21.15.105:3300/0 conn(0x7ff3f40ce420 0x7ff3f415fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:03:49.618 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f2ffd700 1 --2- 172.21.15.170:0/3443609036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ad990 0x7ff3f415be90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4006fd0 tx=0x7ff3e4009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af41941137bc40df server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f1ffb700 1 -- 172.21.15.170:0/3443609036 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4005e30 con 0x7ff3f40ad990 2022-11-27T16:03:49.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f1ffb700 1 -- 172.21.15.170:0/3443609036 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4004030 con 0x7ff3f40ad990 2022-11-27T16:03:49.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.609+0000 7ff3f1ffb700 1 -- 172.21.15.170:0/3443609036 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40043e0 con 0x7ff3f40ad990 2022-11-27T16:03:49.619 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 -- 172.21.15.170:0/3443609036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ad990 msgr2=0x7ff3f415be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 --2- 172.21.15.170:0/3443609036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ad990 0x7ff3f415be90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4006fd0 tx=0x7ff3e4009bd0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 -- 172.21.15.170:0/3443609036 shutdown_connections 2022-11-27T16:03:49.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 --2- 172.21.15.170:0/3443609036 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ad990 0x7ff3f415be90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.620 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 --2- 172.21.15.170:0/3443609036 >> v2:172.21.15.105:3300/0 conn(0x7ff3f40ce420 0x7ff3f415fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 -- 172.21.15.170:0/3443609036 >> 172.21.15.170:0/3443609036 conn(0x7ff3f4070a30 msgr2=0x7ff3f4070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 -- 172.21.15.170:0/3443609036 shutdown_connections 2022-11-27T16:03:49.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.610+0000 7ff3f9838700 1 -- 172.21.15.170:0/3443609036 wait complete. 2022-11-27T16:03:49.621 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.611+0000 7ff3f9838700 1 Processor -- start 2022-11-27T16:03:49.622 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.611+0000 7ff3f9838700 1 -- start start 2022-11-27T16:03:49.625 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f9838700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff3f40ad990 0x7ff3f4097a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.625 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f9838700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ce420 0x7ff3f415df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.626 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f9838700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f41601f0 0x7ff3f415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.626 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f9838700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f41f2010 con 0x7ff3f41601f0 2022-11-27T16:03:49.627 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f9838700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f41f2e70 con 0x7ff3f40ce420 2022-11-27T16:03:49.627 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f9838700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3f41f2260 con 0x7ff3f40ad990 2022-11-27T16:03:49.628 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f27fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ce420 0x7ff3f415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.628 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f2ffd700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff3f40ad990 0x7ff3f4097a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.629 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f27fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ce420 0x7ff3f415df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:49354/0 (socket says 172.21.15.170:49354) 2022-11-27T16:03:49.629 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.612+0000 7ff3f27fc700 1 -- 172.21.15.170:0/4066672196 learned_addr learned my addr 172.21.15.170:0/4066672196 (peer_addr_for_me v2:172.21.15.170:0/0) 2022-11-27T16:03:49.630 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f37fe700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f41601f0 0x7ff3f415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.630 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f27fc700 1 -- 172.21.15.170:0/4066672196 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff3f40ad990 msgr2=0x7ff3f4097a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.630 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f27fc700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff3f40ad990 0x7ff3f4097a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.631 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f27fc700 1 -- 172.21.15.170:0/4066672196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f41601f0 msgr2=0x7ff3f415e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.631 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f27fc700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f41601f0 0x7ff3f415e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.632 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f27fc700 1 -- 172.21.15.170:0/4066672196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e4009500 con 0x7ff3f40ce420 2022-11-27T16:03:49.632 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f27fc700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ce420 0x7ff3f415df80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800e810 tx=0x7ff3e800eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.633 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3dbfff700 1 -- 172.21.15.170:0/4066672196 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800c070 con 0x7ff3f40ce420 2022-11-27T16:03:49.633 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3dbfff700 1 -- 172.21.15.170:0/4066672196 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e8004230 con 0x7ff3f40ce420 2022-11-27T16:03:49.633 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3f41f2540 con 0x7ff3f40ce420 2022-11-27T16:03:49.633 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3dbfff700 1 -- 172.21.15.170:0/4066672196 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80036a0 con 0x7ff3f40ce420 2022-11-27T16:03:49.634 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3f41f2b00 con 0x7ff3f40ce420 2022-11-27T16:03:49.634 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3dbfff700 1 -- 172.21.15.170:0/4066672196 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff3e8019020 con 0x7ff3f40ce420 2022-11-27T16:03:49.634 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.613+0000 7ff3dbfff700 1 -- 172.21.15.170:0/4066672196 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7ff3e8013410 con 0x7ff3f40ce420 2022-11-27T16:03:49.634 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.614+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3f41f2fe0 con 0x7ff3f40ce420 2022-11-27T16:03:49.635 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.616+0000 7ff3dbfff700 1 -- 172.21.15.170:0/4066672196 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7ff3e800f040 con 0x7ff3f40ce420 2022-11-27T16:03:49.700 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.698+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9a2592f7-a19d-49df-beb3-004cd10cf16b", "id": 7} v 0) v1 -- 0x7ff3f4067420 con 0x7ff3f40ce420 2022-11-27T16:03:49.707 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.705+0000 7ff3dbfff700 1 -- 172.21.15.170:0/4066672196 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a2592f7-a19d-49df-beb3-004cd10cf16b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff3e8003bc0 con 0x7ff3f40ce420 2022-11-27T16:03:49.708 INFO:teuthology.orchestra.run.smithi170.stdout:7 2022-11-27T16:03:49.709 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ce420 msgr2=0x7ff3f415df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.710 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ce420 0x7ff3f415df80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800e810 tx=0x7ff3e800eb20 comp rx=0 tx=0).stop 2022-11-27T16:03:49.710 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 shutdown_connections 2022-11-27T16:03:49.711 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff3f40ad990 0x7ff3f4097a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.711 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3f41601f0 0x7ff3f415e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.711 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 --2- 172.21.15.170:0/4066672196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3f40ce420 0x7ff3f415df80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.712 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 >> 172.21.15.170:0/4066672196 conn(0x7ff3f4070a30 msgr2=0x7ff3f40ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.712 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 shutdown_connections 2022-11-27T16:03:49.712 INFO:teuthology.orchestra.run.smithi170.stderr:2022-11-27T16:03:49.706+0000 7ff3f9838700 1 -- 172.21.15.170:0/4066672196 wait complete. 2022-11-27T16:03:49.720 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T16:03:49.721 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T16:03:49.723 INFO:tasks.ceph.osd.0:Started 2022-11-27T16:03:49.724 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T16:03:49.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T16:03:49.726 INFO:tasks.ceph.osd.1:Started 2022-11-27T16:03:49.727 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T16:03:49.727 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T16:03:49.731 INFO:tasks.ceph.osd.2:Started 2022-11-27T16:03:49.732 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T16:03:49.733 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T16:03:49.736 INFO:tasks.ceph.osd.3:Started 2022-11-27T16:03:49.737 INFO:tasks.ceph.osd.4:Restarting daemon 2022-11-27T16:03:49.738 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-11-27T16:03:49.741 INFO:tasks.ceph.osd.4:Started 2022-11-27T16:03:49.742 INFO:tasks.ceph.osd.5:Restarting daemon 2022-11-27T16:03:49.743 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-11-27T16:03:49.746 INFO:tasks.ceph.osd.5:Started 2022-11-27T16:03:49.746 INFO:tasks.ceph.osd.6:Restarting daemon 2022-11-27T16:03:49.747 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-11-27T16:03:49.750 INFO:tasks.ceph.osd.6:Started 2022-11-27T16:03:49.751 INFO:tasks.ceph.osd.7:Restarting daemon 2022-11-27T16:03:49.752 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-11-27T16:03:49.755 INFO:tasks.ceph.osd.7:Started 2022-11-27T16:03:49.756 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:03:49.857 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:49.856+0000 7fad4d8f2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:03:49.898 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.897+0000 7f640053e700 1 Processor -- start 2022-11-27T16:03:49.899 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.897+0000 7f640053e700 1 -- start start 2022-11-27T16:03:49.902 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f640053e700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63f8070f60 0x7f63f806a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.902 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f640053e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f63f8071f60 0x7f63f80723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.902 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f640053e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f8072900 con 0x7f63f815be80 2022-11-27T16:03:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f640053e700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f8070b40 con 0x7f63f8070f60 2022-11-27T16:03:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f640053e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f815c250 con 0x7f63f8071f60 2022-11-27T16:03:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63f8070f60 0x7f63f806a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.903 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f63f8070f60 0x7f63f806a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42452/0 (socket says 172.21.15.105:42452) 2022-11-27T16:03:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 -- 172.21.15.105:0/1664239035 learned_addr learned my addr 172.21.15.105:0/1664239035 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63feadb700 1 --2- 172.21.15.105:0/1664239035 >> v2:172.21.15.170:3300/0 conn(0x7f63f8071f60 0x7f63f80723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 -- 172.21.15.105:0/1664239035 >> v2:172.21.15.170:3300/0 conn(0x7f63f8071f60 msgr2=0x7f63f80723c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:49.904 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 --2- 172.21.15.105:0/1664239035 >> v2:172.21.15.170:3300/0 conn(0x7f63f8071f60 0x7f63f80723c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 -- 172.21.15.105:0/1664239035 >> v1:172.21.15.170:6789/0 conn(0x7f63f815be80 legacy=0x7f63f80cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 -- 172.21.15.105:0/1664239035 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f80ce580 con 0x7f63f8070f60 2022-11-27T16:03:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63feadb700 1 --2- 172.21.15.105:0/1664239035 >> v2:172.21.15.170:3300/0 conn(0x7f63f8071f60 0x7f63f80723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:03:49.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.898+0000 7f63fdad9700 1 --2- 172.21.15.105:0/1664239035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f8070f60 0x7f63f806a9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f63f8071b20 tx=0x7f63e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0edefe6c68cfe64 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f63fd2d8700 1 -- 172.21.15.105:0/1664239035 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800b870 con 0x7f63f8070f60 2022-11-27T16:03:49.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f63fd2d8700 1 -- 172.21.15.105:0/1664239035 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e800b9d0 con 0x7f63f8070f60 2022-11-27T16:03:49.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f63fd2d8700 1 -- 172.21.15.105:0/1664239035 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e800bd60 con 0x7f63f8070f60 2022-11-27T16:03:49.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 -- 172.21.15.105:0/1664239035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f8070f60 msgr2=0x7f63f806a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 --2- 172.21.15.105:0/1664239035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f8070f60 0x7f63f806a9b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f63f8071b20 tx=0x7f63e8009bd0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 -- 172.21.15.105:0/1664239035 shutdown_connections 2022-11-27T16:03:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 --2- 172.21.15.105:0/1664239035 >> v2:172.21.15.170:3300/0 conn(0x7f63f8071f60 0x7f63f80723c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.907 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 --2- 172.21.15.105:0/1664239035 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f8070f60 0x7f63f806a9b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.908 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 -- 172.21.15.105:0/1664239035 >> 172.21.15.105:0/1664239035 conn(0x7f63f80cb7e0 msgr2=0x7f63f80cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.908 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 -- 172.21.15.105:0/1664239035 shutdown_connections 2022-11-27T16:03:49.908 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.899+0000 7f640053e700 1 -- 172.21.15.105:0/1664239035 wait complete. 2022-11-27T16:03:49.908 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.901+0000 7f640053e700 1 Processor -- start 2022-11-27T16:03:49.909 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.901+0000 7f640053e700 1 -- start start 2022-11-27T16:03:49.911 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f640053e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63f8070f60 0x7f63f814f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.912 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f640053e700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8071f60 0x7f63f81f1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.912 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f640053e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f815be80 0x7f63f80b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:49.912 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f640053e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81f1670 con 0x7f63f815be80 2022-11-27T16:03:49.912 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f640053e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f814cee0 con 0x7f63f8070f60 2022-11-27T16:03:49.912 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f640053e700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f80b4400 con 0x7f63f8071f60 2022-11-27T16:03:49.913 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f815be80 0x7f63f80b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.913 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f815be80 0x7f63f80b3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42460/0 (socket says 172.21.15.105:42460) 2022-11-27T16:03:49.913 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 -- 172.21.15.105:0/1491798824 learned_addr learned my addr 172.21.15.105:0/1491798824 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:49.913 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63fe2da700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63f8070f60 0x7f63f814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.914 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63fdad9700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8071f60 0x7f63f81f1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:49.914 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 -- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8071f60 msgr2=0x7f63f81f1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.914 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8071f60 0x7f63f81f1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.914 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 -- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63f8070f60 msgr2=0x7f63f814f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.914 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63f8070f60 0x7f63f814f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 -- 172.21.15.105:0/1491798824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e8009550 con 0x7f63f815be80 2022-11-27T16:03:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.902+0000 7f63feadb700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f815be80 0x7f63f80b3bb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63f0007f20 tx=0x7f63f000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.903+0000 7f63e6ffd700 1 -- 172.21.15.105:0/1491798824 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000dd80 con 0x7f63f815be80 2022-11-27T16:03:49.915 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.903+0000 7f63e6ffd700 1 -- 172.21.15.105:0/1491798824 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f000f040 con 0x7f63f815be80 2022-11-27T16:03:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.903+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f80b46e0 con 0x7f63f815be80 2022-11-27T16:03:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.903+0000 7f63e6ffd700 1 -- 172.21.15.105:0/1491798824 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000dee0 con 0x7f63f815be80 2022-11-27T16:03:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.903+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f80b40f0 con 0x7f63f815be80 2022-11-27T16:03:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.903+0000 7f63e6ffd700 1 -- 172.21.15.105:0/1491798824 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f63f001c030 con 0x7f63f815be80 2022-11-27T16:03:49.916 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.903+0000 7f63e6ffd700 1 -- 172.21.15.105:0/1491798824 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f63f0004710 con 0x7f63f815be80 2022-11-27T16:03:49.917 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.904+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f814d390 con 0x7f63f815be80 2022-11-27T16:03:49.917 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.906+0000 7f63e6ffd700 1 -- 172.21.15.105:0/1491798824 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f63f814d390 con 0x7f63f815be80 2022-11-27T16:03:49.993 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.992+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63f8067420 con 0x7f63f815be80 2022-11-27T16:03:49.994 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f63e6ffd700 1 -- 172.21.15.105:0/1491798824 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f63f001a070 con 0x7f63f815be80 2022-11-27T16:03:49.994 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:03:49.995 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":10,"fsid":"558b439a-8d62-4391-a123-f6ff0625ded9","created":"2022-11-27T16:03:46.292863+0000","modified":"2022-11-27T16:03:49.700155+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:03:49.700155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"a877ed0a-6953-4642-9d56-f1eae305b6a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dae5370d-59b3-458c-9821-f88f47769ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e34db7ee-5ac2-4682-8b3d-77781aff8feb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"98f2fda2-a32e-4e5c-ad39-c3d67013ccd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ec8582a4-93d8-4e63-8b94-5386bf561c11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"55b1b3a8-5dac-4f1e-b0d6-1871422ae097","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"64ff9585-481c-4ab8-a276-dad128c5dba9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9a2592f7-a19d-49df-beb3-004cd10cf16b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:03:49.996 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f815be80 msgr2=0x7f63f80b3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:49.996 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f640053e700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f815be80 0x7f63f80b3bb0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f63f0007f20 tx=0x7f63f000d3b0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.996 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 shutdown_connections 2022-11-27T16:03:49.997 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f640053e700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f63f815be80 0x7f63f80b3bb0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.997 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f640053e700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f63f8070f60 0x7f63f814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.997 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f640053e700 1 --2- 172.21.15.105:0/1491798824 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f63f8071f60 0x7f63f81f1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:49.997 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.993+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 >> 172.21.15.105:0/1491798824 conn(0x7f63f80cb7e0 msgr2=0x7f63f81e6270 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:49.997 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.994+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 shutdown_connections 2022-11-27T16:03:49.998 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:49.994+0000 7f640053e700 1 -- 172.21.15.105:0/1491798824 wait complete. 2022-11-27T16:03:49.998 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:49.993+0000 7f90de590200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:03:50.007 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T16:03:50.008 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T16:03:50.090 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:50.088+0000 7fad4d8f2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:03:50.102 INFO:tasks.ceph.osd.0.smithi105.stderr:2022-11-27T16:03:50.101+0000 7fcfb8d345c0 -1 Falling back to public interface 2022-11-27T16:03:50.107 INFO:tasks.ceph.osd.1.smithi105.stderr:2022-11-27T16:03:50.106+0000 7f5fbd6a25c0 -1 Falling back to public interface 2022-11-27T16:03:50.206 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:50.204+0000 7f90de590200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:03:50.358 INFO:tasks.ceph.osd.4.smithi170.stderr:2022-11-27T16:03:50.356+0000 7f33e5f6b5c0 -1 Falling back to public interface 2022-11-27T16:03:50.360 INFO:tasks.ceph.osd.2.smithi105.stderr:2022-11-27T16:03:50.359+0000 7ff27b1565c0 -1 Falling back to public interface 2022-11-27T16:03:50.363 INFO:tasks.ceph.osd.6.smithi170.stderr:2022-11-27T16:03:50.360+0000 7f87417535c0 -1 Falling back to public interface 2022-11-27T16:03:50.372 INFO:tasks.ceph.osd.3.smithi105.stderr:2022-11-27T16:03:50.371+0000 7fb7e4a245c0 -1 Falling back to public interface 2022-11-27T16:03:50.376 INFO:tasks.ceph.osd.5.smithi170.stderr:2022-11-27T16:03:50.374+0000 7f1bcdbfd5c0 -1 Falling back to public interface 2022-11-27T16:03:50.377 INFO:tasks.ceph.osd.7.smithi170.stderr:2022-11-27T16:03:50.374+0000 7f2f893275c0 -1 Falling back to public interface 2022-11-27T16:03:50.815 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:03:50.842 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:50.841+0000 7fad4d8f2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:03:50.915 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:50.914+0000 7fad4d8f2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:03:50.930 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.928+0000 7f2959fd9700 1 Processor -- start 2022-11-27T16:03:50.930 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f2959fd9700 1 -- start start 2022-11-27T16:03:50.932 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f2959fd9700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29540b4a40 0x7f295406bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:50.933 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f2959fd9700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29540ca840 con 0x7f29540729e0 2022-11-27T16:03:50.933 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f2959fd9700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29540ca530 con 0x7f295406c490 2022-11-27T16:03:50.933 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f2959fd9700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29540b4e90 con 0x7f29540b4a40 2022-11-27T16:03:50.934 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f2952ffd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f29540b4a40 0x7f295406bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:50.935 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f29537fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f29540729e0 0x7f2954159c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45760/0 (socket says 172.21.15.105:45760) 2022-11-27T16:03:50.935 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.929+0000 7f29537fe700 1 -- 172.21.15.105:0/4245678048 learned_addr learned my addr 172.21.15.105:0/4245678048 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:03:50.936 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f2952ffd700 1 -- 172.21.15.105:0/4245678048 >> v1:172.21.15.170:6789/0 conn(0x7f295406c490 legacy=0x7f29540cacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:50.936 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f2952ffd700 1 -- 172.21.15.105:0/4245678048 >> v1:172.21.15.105:6789/0 conn(0x7f29540729e0 legacy=0x7f2954159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:50.936 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f2952ffd700 1 -- 172.21.15.105:0/4245678048 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295415a410 con 0x7f29540b4a40 2022-11-27T16:03:50.936 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f29527fc700 1 -- 172.21.15.105:0/4245678048 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794848775 0 0) 0x7f29540ca840 con 0x7f29540729e0 2022-11-27T16:03:50.937 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f2952ffd700 1 --2- 172.21.15.105:0/4245678048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29540b4a40 0x7f295406bf50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f293c009840 tx=0x7f293c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913cf0a6aac52426 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:50.937 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f29527fc700 1 -- 172.21.15.105:0/4245678048 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c011830 con 0x7f29540b4a40 2022-11-27T16:03:50.937 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f29527fc700 1 -- 172.21.15.105:0/4245678048 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f293c011990 con 0x7f29540b4a40 2022-11-27T16:03:50.937 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.930+0000 7f29527fc700 1 -- 172.21.15.105:0/4245678048 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c011d20 con 0x7f29540b4a40 2022-11-27T16:03:50.937 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.931+0000 7f2959fd9700 1 -- 172.21.15.105:0/4245678048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29540b4a40 msgr2=0x7f295406bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:50.938 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.931+0000 7f2959fd9700 1 --2- 172.21.15.105:0/4245678048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29540b4a40 0x7f295406bf50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f293c009840 tx=0x7f293c009b50 comp rx=0 tx=0).stop 2022-11-27T16:03:50.938 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.931+0000 7f2959fd9700 1 -- 172.21.15.105:0/4245678048 shutdown_connections 2022-11-27T16:03:50.938 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.931+0000 7f2959fd9700 1 --2- 172.21.15.105:0/4245678048 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29540b4a40 0x7f295406bf50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:50.938 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.931+0000 7f2959fd9700 1 -- 172.21.15.105:0/4245678048 >> 172.21.15.105:0/4245678048 conn(0x7f29540b1640 msgr2=0x7f29540cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:50.939 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.931+0000 7f2959fd9700 1 -- 172.21.15.105:0/4245678048 shutdown_connections 2022-11-27T16:03:50.939 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.931+0000 7f2959fd9700 1 -- 172.21.15.105:0/4245678048 wait complete. 2022-11-27T16:03:50.942 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.932+0000 7f2959fd9700 1 Processor -- start 2022-11-27T16:03:50.942 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.932+0000 7f2959fd9700 1 -- start start 2022-11-27T16:03:50.942 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2959fd9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f295406c490 0x7f29541f0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:50.943 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2959fd9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29540729e0 0x7f295414a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:50.943 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2959fd9700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29540b4a40 0x7f29540ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:50.943 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2959fd9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295414ad30 con 0x7f295406c490 2022-11-27T16:03:50.943 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2959fd9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2954149d60 con 0x7f29540729e0 2022-11-27T16:03:50.943 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2959fd9700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29540ac220 con 0x7f29540b4a40 2022-11-27T16:03:50.944 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f295406c490 0x7f29541f0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:50.944 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f295406c490 0x7f29541f0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50786/0 (socket says 172.21.15.105:50786) 2022-11-27T16:03:50.945 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 -- 172.21.15.105:0/2327359468 learned_addr learned my addr 172.21.15.105:0/2327359468 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:50.945 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2952ffd700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29540729e0 0x7f295414a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:50.945 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f2953fff700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29540b4a40 0x7f29540ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:50.945 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 -- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29540b4a40 msgr2=0x7f29540ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:50.946 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29540b4a40 0x7f29540ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:50.946 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 -- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29540729e0 msgr2=0x7f295414a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:50.946 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29540729e0 0x7f295414a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:50.946 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 -- 172.21.15.105:0/2327359468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c0094a0 con 0x7f295406c490 2022-11-27T16:03:50.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.933+0000 7f29537fe700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f295406c490 0x7f29541f0ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f294400c520 tx=0x7f294400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:50.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.934+0000 7f294bfff700 1 -- 172.21.15.105:0/2327359468 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944004c80 con 0x7f295406c490 2022-11-27T16:03:50.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.934+0000 7f294bfff700 1 -- 172.21.15.105:0/2327359468 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2944004e00 con 0x7f295406c490 2022-11-27T16:03:50.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.934+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29540ac500 con 0x7f295406c490 2022-11-27T16:03:50.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.934+0000 7f294bfff700 1 -- 172.21.15.105:0/2327359468 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944007310 con 0x7f295406c490 2022-11-27T16:03:50.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.934+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295414a2d0 con 0x7f295406c490 2022-11-27T16:03:50.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.934+0000 7f294bfff700 1 -- 172.21.15.105:0/2327359468 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f294401a020 con 0x7f295406c490 2022-11-27T16:03:50.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.934+0000 7f294bfff700 1 -- 172.21.15.105:0/2327359468 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f2944013a90 con 0x7f295406c490 2022-11-27T16:03:50.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.935+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295414a4c0 con 0x7f295406c490 2022-11-27T16:03:50.949 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:50.937+0000 7f294bfff700 1 -- 172.21.15.105:0/2327359468 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f294401d4a0 con 0x7f295406c490 2022-11-27T16:03:50.962 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:50.959+0000 7f90de590200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:03:51.007 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.006+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2954067420 con 0x7f295406c490 2022-11-27T16:03:51.008 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.007+0000 7f294bfff700 1 -- 172.21.15.105:0/2327359468 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f2944013dd0 con 0x7f295406c490 2022-11-27T16:03:51.009 INFO:teuthology.misc.health.smithi105.stdout: 2022-11-27T16:03:51.009 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":10,"fsid":"558b439a-8d62-4391-a123-f6ff0625ded9","created":"2022-11-27T16:03:46.292863+0000","modified":"2022-11-27T16:03:49.700155+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:03:49.700155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"a877ed0a-6953-4642-9d56-f1eae305b6a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"dae5370d-59b3-458c-9821-f88f47769ab8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e34db7ee-5ac2-4682-8b3d-77781aff8feb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"98f2fda2-a32e-4e5c-ad39-c3d67013ccd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ec8582a4-93d8-4e63-8b94-5386bf561c11","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"55b1b3a8-5dac-4f1e-b0d6-1871422ae097","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"64ff9585-481c-4ab8-a276-dad128c5dba9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9a2592f7-a19d-49df-beb3-004cd10cf16b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:03:51.010 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f295406c490 msgr2=0x7f29541f0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:51.011 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f295406c490 0x7f29541f0ca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f294400c520 tx=0x7f294400c830 comp rx=0 tx=0).stop 2022-11-27T16:03:51.011 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 shutdown_connections 2022-11-27T16:03:51.011 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f29540b4a40 0x7f29540ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:51.011 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f295406c490 0x7f29541f0ca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:51.012 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 --2- 172.21.15.105:0/2327359468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f29540729e0 0x7f295414a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:51.012 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 >> 172.21.15.105:0/2327359468 conn(0x7f29540b1640 msgr2=0x7f2954159400 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:51.012 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 shutdown_connections 2022-11-27T16:03:51.012 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:51.008+0000 7f2959fd9700 1 -- 172.21.15.105:0/2327359468 wait complete. 2022-11-27T16:03:51.020 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T16:03:51.031 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:51.029+0000 7f90de590200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:03:51.445 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:51.444+0000 7fad4d8f2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:03:51.525 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:51.524+0000 7fad4d8f2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:03:51.551 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:51.549+0000 7f90de590200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:03:51.634 INFO:tasks.ceph.mgr.x.smithi170.stderr:2022-11-27T16:03:51.631+0000 7f90de590200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:03:52.549 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:52.548+0000 7fad0b659700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-11-27T16:03:52.549 INFO:tasks.ceph.mgr.y.smithi105.stderr:2022-11-27T16:03:52.548+0000 7fad0b659700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-11-27T16:03:53.591 INFO:tasks.ceph.osd.1.smithi105.stderr:2022-11-27T16:03:53.589+0000 7f5fbd6a25c0 -1 osd.1 0 log_to_monitors true 2022-11-27T16:03:53.601 INFO:tasks.ceph.osd.0.smithi105.stderr:2022-11-27T16:03:53.599+0000 7fcfb8d345c0 -1 osd.0 0 log_to_monitors true 2022-11-27T16:03:53.803 INFO:tasks.ceph.osd.2.smithi105.stderr:2022-11-27T16:03:53.801+0000 7ff27b1565c0 -1 osd.2 0 log_to_monitors true 2022-11-27T16:03:53.809 INFO:tasks.ceph.osd.6.smithi170.stderr:2022-11-27T16:03:53.807+0000 7f87417535c0 -1 osd.6 0 log_to_monitors true 2022-11-27T16:03:53.861 INFO:tasks.ceph.osd.4.smithi170.stderr:2022-11-27T16:03:53.859+0000 7f33e5f6b5c0 -1 osd.4 0 log_to_monitors true 2022-11-27T16:03:53.886 INFO:tasks.ceph.osd.5.smithi170.stderr:2022-11-27T16:03:53.883+0000 7f1bcdbfd5c0 -1 osd.5 0 log_to_monitors true 2022-11-27T16:03:53.901 INFO:tasks.ceph.osd.7.smithi170.stderr:2022-11-27T16:03:53.899+0000 7f2f893275c0 -1 osd.7 0 log_to_monitors true 2022-11-27T16:03:53.926 INFO:tasks.ceph.osd.3.smithi105.stderr:2022-11-27T16:03:53.924+0000 7fb7e4a245c0 -1 osd.3 0 log_to_monitors true 2022-11-27T16:03:56.850 INFO:tasks.ceph.osd.5.smithi170.stderr:2022-11-27T16:03:56.847+0000 7f1bc17a5700 -1 osd.5 0 waiting for initial osdmap 2022-11-27T16:03:56.868 INFO:tasks.ceph.osd.5.smithi170.stderr:2022-11-27T16:03:56.866+0000 7f1bb9b80700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:56.869 INFO:tasks.ceph.osd.4.smithi170.stderr:2022-11-27T16:03:56.867+0000 7f33d9b13700 -1 osd.4 0 waiting for initial osdmap 2022-11-27T16:03:56.880 INFO:tasks.ceph.osd.6.smithi170.stderr:2022-11-27T16:03:56.878+0000 7f87352fb700 -1 osd.6 0 waiting for initial osdmap 2022-11-27T16:03:56.882 INFO:tasks.ceph.osd.4.smithi170.stderr:2022-11-27T16:03:56.880+0000 7f33d1eee700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:56.886 INFO:tasks.ceph.osd.7.smithi170.stderr:2022-11-27T16:03:56.885+0000 7f2f7cecf700 -1 osd.7 0 waiting for initial osdmap 2022-11-27T16:03:56.893 INFO:tasks.ceph.osd.6.smithi170.stderr:2022-11-27T16:03:56.891+0000 7f872d6d6700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:56.898 INFO:tasks.ceph.osd.7.smithi170.stderr:2022-11-27T16:03:56.896+0000 7f2f752aa700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:57.572 INFO:tasks.ceph.osd.3.smithi105.stderr:2022-11-27T16:03:57.570+0000 7fb7d85cc700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T16:03:57.584 INFO:tasks.ceph.osd.3.smithi105.stderr:2022-11-27T16:03:57.582+0000 7fb7d09a7700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:57.605 INFO:tasks.ceph.osd.0.smithi105.stderr:2022-11-27T16:03:57.603+0000 7fcfac8dc700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T16:03:57.619 INFO:tasks.ceph.osd.1.smithi105.stderr:2022-11-27T16:03:57.617+0000 7f5fb124a700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T16:03:57.619 INFO:tasks.ceph.osd.0.smithi105.stderr:2022-11-27T16:03:57.618+0000 7fcfa4cb7700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:57.632 INFO:tasks.ceph.osd.2.smithi105.stderr:2022-11-27T16:03:57.630+0000 7ff26ecfe700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T16:03:57.668 INFO:tasks.ceph.osd.1.smithi105.stderr:2022-11-27T16:03:57.632+0000 7f5fa9625700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:57.668 INFO:tasks.ceph.osd.2.smithi105.stderr:2022-11-27T16:03:57.651+0000 7ff2670d9700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:03:57.827 DEBUG:teuthology.orchestra.run.smithi105:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:03:57.944 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.942+0000 7f99b01af700 1 Processor -- start 2022-11-27T16:03:57.944 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.942+0000 7f99b01af700 1 -- start start 2022-11-27T16:03:57.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99b01af700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f99a806d6c0 0x7f99a8166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:57.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99b01af700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a80af3f0 con 0x7f99a80b2640 2022-11-27T16:03:57.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99b01af700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a815fce0 con 0x7f99a80b1b60 2022-11-27T16:03:57.947 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99b01af700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a815ffc0 con 0x7f99a806d6c0 2022-11-27T16:03:57.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99ae74c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f99a806d6c0 0x7f99a8166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:57.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99ad74a700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f99a80b2640 0x7f99a806cfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45766/0 (socket says 172.21.15.105:45766) 2022-11-27T16:03:57.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99ad74a700 1 -- 172.21.15.105:0/3628625316 learned_addr learned my addr 172.21.15.105:0/3628625316 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:03:57.948 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99ae74c700 1 -- 172.21.15.105:0/3628625316 >> v1:172.21.15.170:6789/0 conn(0x7f99a80b1b60 legacy=0x7f99a80b1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:57.949 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99ae74c700 1 -- 172.21.15.105:0/3628625316 >> v1:172.21.15.105:6789/0 conn(0x7f99a80b2640 legacy=0x7f99a806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:57.949 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99ae74c700 1 -- 172.21.15.105:0/3628625316 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a81602a0 con 0x7f99a806d6c0 2022-11-27T16:03:57.949 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99acf49700 1 -- 172.21.15.105:0/3628625316 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418670583 0 0) 0x7f99a80af3f0 con 0x7f99a80b2640 2022-11-27T16:03:57.949 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99ae74c700 1 --2- 172.21.15.105:0/3628625316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a806d6c0 0x7f99a8166aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99a80af5c0 tx=0x7f99a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662fc8cc7d9261ad server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:57.949 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.943+0000 7f99acf49700 1 -- 172.21.15.105:0/3628625316 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0016070 con 0x7f99a806d6c0 2022-11-27T16:03:57.950 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99acf49700 1 -- 172.21.15.105:0/3628625316 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a000bbd0 con 0x7f99a806d6c0 2022-11-27T16:03:57.950 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99acf49700 1 -- 172.21.15.105:0/3628625316 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a000e070 con 0x7f99a806d6c0 2022-11-27T16:03:57.950 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99b01af700 1 -- 172.21.15.105:0/3628625316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a806d6c0 msgr2=0x7f99a8166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:57.950 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99b01af700 1 --2- 172.21.15.105:0/3628625316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a806d6c0 0x7f99a8166aa0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f99a80af5c0 tx=0x7f99a000b330 comp rx=0 tx=0).stop 2022-11-27T16:03:57.951 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99b01af700 1 -- 172.21.15.105:0/3628625316 shutdown_connections 2022-11-27T16:03:57.951 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99b01af700 1 --2- 172.21.15.105:0/3628625316 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a806d6c0 0x7f99a8166aa0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:57.951 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99b01af700 1 -- 172.21.15.105:0/3628625316 >> 172.21.15.105:0/3628625316 conn(0x7f99a80cc9d0 msgr2=0x7f99a80ccdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:57.951 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99b01af700 1 -- 172.21.15.105:0/3628625316 shutdown_connections 2022-11-27T16:03:57.952 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.944+0000 7f99b01af700 1 -- 172.21.15.105:0/3628625316 wait complete. 2022-11-27T16:03:57.956 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.945+0000 7f99b01af700 1 Processor -- start 2022-11-27T16:03:57.956 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.945+0000 7f99b01af700 1 -- start start 2022-11-27T16:03:57.957 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99b01af700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a806d6c0 0x7f99a80c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:57.957 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99b01af700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a80b1b60 0x7f99a80ace50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:57.957 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99b01af700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a80b2640 0x7f99a80ad390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:57.957 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99b01af700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a81475b0 con 0x7f99a80b1b60 2022-11-27T16:03:57.958 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99b01af700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8146c10 con 0x7f99a806d6c0 2022-11-27T16:03:57.958 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99b01af700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a8146ef0 con 0x7f99a80b2640 2022-11-27T16:03:57.958 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a80b1b60 0x7f99a80ace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:57.959 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a80b1b60 0x7f99a80ace50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50888/0 (socket says 172.21.15.105:50888) 2022-11-27T16:03:57.959 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 -- 172.21.15.105:0/1523802251 learned_addr learned my addr 172.21.15.105:0/1523802251 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:57.959 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ae74c700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a80b2640 0x7f99a80ad390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:57.959 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99adf4b700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a806d6c0 0x7f99a80c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:57.960 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 -- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a80b2640 msgr2=0x7f99a80ad390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:57.960 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a80b2640 0x7f99a80ad390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:57.960 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 -- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a806d6c0 msgr2=0x7f99a80c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:57.961 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a806d6c0 0x7f99a80c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:57.961 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 -- 172.21.15.105:0/1523802251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0009a70 con 0x7f99a80b1b60 2022-11-27T16:03:57.961 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ae74c700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a80b2640 0x7f99a80ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:03:57.961 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.946+0000 7f99ad74a700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a80b1b60 0x7f99a80ace50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9998001eb0 tx=0x7f999800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:57.961 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.947+0000 7f99967fc700 1 -- 172.21.15.105:0/1523802251 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998004c80 con 0x7f99a80b1b60 2022-11-27T16:03:57.962 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.947+0000 7f99967fc700 1 -- 172.21.15.105:0/1523802251 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998004de0 con 0x7f99a80b1b60 2022-11-27T16:03:57.962 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.947+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a81471d0 con 0x7f99a80b1b60 2022-11-27T16:03:57.963 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.947+0000 7f99967fc700 1 -- 172.21.15.105:0/1523802251 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998007310 con 0x7f99a80b1b60 2022-11-27T16:03:57.963 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.947+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a80c6880 con 0x7f99a80b1b60 2022-11-27T16:03:57.963 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.948+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a8067420 con 0x7f99a80b1b60 2022-11-27T16:03:57.964 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.948+0000 7f99967fc700 1 -- 172.21.15.105:0/1523802251 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7f999801a020 con 0x7f99a80b1b60 2022-11-27T16:03:57.964 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.948+0000 7f99967fc700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f9988066540 0x7f99880689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:57.965 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.948+0000 7f99adf4b700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f9988066540 0x7f99880689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:57.965 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.948+0000 7f99967fc700 1 -- 172.21.15.105:0/1523802251 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7f9998082d30 con 0x7f99a80b1b60 2022-11-27T16:03:57.966 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.949+0000 7f99adf4b700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f9988066540 0x7f99880689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000f60 tx=0x7f99a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:57.966 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:57.953+0000 7f99967fc700 1 -- 172.21.15.105:0/1523802251 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f9998050ac0 con 0x7f99a80b1b60 2022-11-27T16:03:58.132 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.130+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99a80cc3a0 con 0x7f99a80b1b60 2022-11-27T16:03:58.133 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.131+0000 7f99967fc700 1 -- 172.21.15.105:0/1523802251 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10538 (secure 0 0 0) 0x7f99980540e0 con 0x7f99a80b1b60 2022-11-27T16:03:58.134 INFO:teuthology.misc.health.smithi105.stdout: 2022-11-27T16:03:58.134 INFO:teuthology.misc.health.smithi105.stdout:{"epoch":13,"fsid":"558b439a-8d62-4391-a123-f6ff0625ded9","created":"2022-11-27T16:03:46.292863+0000","modified":"2022-11-27T16:03:57.895226+0000","last_up_change":"2022-11-27T16:03:57.895226+0000","last_in_change":"2022-11-27T16:03:49.700155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"a877ed0a-6953-4642-9d56-f1eae305b6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":112238},{"type":"v1","addr":"172.21.15.105:6801","nonce":112238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":112238},{"type":"v1","addr":"172.21.15.105:6803","nonce":112238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":112238},{"type":"v1","addr":"172.21.15.105:6807","nonce":112238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":112238},{"type":"v1","addr":"172.21.15.105:6805","nonce":112238}]},"public_addr":"172.21.15.105:6801/112238","cluster_addr":"172.21.15.105:6803/112238","heartbeat_back_addr":"172.21.15.105:6807/112238","heartbeat_front_addr":"172.21.15.105:6805/112238","state":["exists","up"]},{"osd":1,"uuid":"dae5370d-59b3-458c-9821-f88f47769ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":112239},{"type":"v1","addr":"172.21.15.105:6809","nonce":112239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":112239},{"type":"v1","addr":"172.21.15.105:6811","nonce":112239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":112239},{"type":"v1","addr":"172.21.15.105:6815","nonce":112239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":112239},{"type":"v1","addr":"172.21.15.105:6813","nonce":112239}]},"public_addr":"172.21.15.105:6809/112239","cluster_addr":"172.21.15.105:6811/112239","heartbeat_back_addr":"172.21.15.105:6815/112239","heartbeat_front_addr":"172.21.15.105:6813/112239","state":["exists","up"]},{"osd":2,"uuid":"e34db7ee-5ac2-4682-8b3d-77781aff8feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":112240},{"type":"v1","addr":"172.21.15.105:6817","nonce":112240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":112240},{"type":"v1","addr":"172.21.15.105:6819","nonce":112240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":112240},{"type":"v1","addr":"172.21.15.105:6823","nonce":112240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":112240},{"type":"v1","addr":"172.21.15.105:6821","nonce":112240}]},"public_addr":"172.21.15.105:6817/112240","cluster_addr":"172.21.15.105:6819/112240","heartbeat_back_addr":"172.21.15.105:6823/112240","heartbeat_front_addr":"172.21.15.105:6821/112240","state":["exists","up"]},{"osd":3,"uuid":"98f2fda2-a32e-4e5c-ad39-c3d67013ccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":112241},{"type":"v1","addr":"172.21.15.105:6825","nonce":112241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":112241},{"type":"v1","addr":"172.21.15.105:6827","nonce":112241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":112241},{"type":"v1","addr":"172.21.15.105:6831","nonce":112241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":112241},{"type":"v1","addr":"172.21.15.105:6829","nonce":112241}]},"public_addr":"172.21.15.105:6825/112241","cluster_addr":"172.21.15.105:6827/112241","heartbeat_back_addr":"172.21.15.105:6831/112241","heartbeat_front_addr":"172.21.15.105:6829/112241","state":["exists","up"]},{"osd":4,"uuid":"ec8582a4-93d8-4e63-8b94-5386bf561c11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":112474},{"type":"v1","addr":"172.21.15.170:6801","nonce":112474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":112474},{"type":"v1","addr":"172.21.15.170:6803","nonce":112474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":112474},{"type":"v1","addr":"172.21.15.170:6807","nonce":112474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":112474},{"type":"v1","addr":"172.21.15.170:6805","nonce":112474}]},"public_addr":"172.21.15.170:6801/112474","cluster_addr":"172.21.15.170:6803/112474","heartbeat_back_addr":"172.21.15.170:6807/112474","heartbeat_front_addr":"172.21.15.170:6805/112474","state":["exists","up"]},{"osd":5,"uuid":"55b1b3a8-5dac-4f1e-b0d6-1871422ae097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":112486},{"type":"v1","addr":"172.21.15.170:6818","nonce":112486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6819","nonce":112486},{"type":"v1","addr":"172.21.15.170:6821","nonce":112486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":112486},{"type":"v1","addr":"172.21.15.170:6827","nonce":112486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":112486},{"type":"v1","addr":"172.21.15.170:6824","nonce":112486}]},"public_addr":"172.21.15.170:6818/112486","cluster_addr":"172.21.15.170:6821/112486","heartbeat_back_addr":"172.21.15.170:6827/112486","heartbeat_front_addr":"172.21.15.170:6824/112486","state":["exists","up"]},{"osd":6,"uuid":"64ff9585-481c-4ab8-a276-dad128c5dba9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":112475},{"type":"v1","addr":"172.21.15.170:6809","nonce":112475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":112475},{"type":"v1","addr":"172.21.15.170:6811","nonce":112475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":112475},{"type":"v1","addr":"172.21.15.170:6815","nonce":112475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":112475},{"type":"v1","addr":"172.21.15.170:6813","nonce":112475}]},"public_addr":"172.21.15.170:6809/112475","cluster_addr":"172.21.15.170:6811/112475","heartbeat_back_addr":"172.21.15.170:6815/112475","heartbeat_front_addr":"172.21.15.170:6813/112475","state":["exists","up"]},{"osd":7,"uuid":"9a2592f7-a19d-49df-beb3-004cd10cf16b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":112495},{"type":"v1","addr":"172.21.15.170:6820","nonce":112495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":112495},{"type":"v1","addr":"172.21.15.170:6825","nonce":112495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":112495},{"type":"v1","addr":"172.21.15.170:6831","nonce":112495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":112495},{"type":"v1","addr":"172.21.15.170:6829","nonce":112495}]},"public_addr":"172.21.15.170:6820/112495","cluster_addr":"172.21.15.170:6825/112495","heartbeat_back_addr":"172.21.15.170:6831/112495","heartbeat_front_addr":"172.21.15.170:6829/112495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:03:58.135 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f9988066540 msgr2=0x7f99880689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.136 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f9988066540 0x7f99880689f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000f60 tx=0x7f99a4009040 comp rx=0 tx=0).stop 2022-11-27T16:03:58.136 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a80b1b60 msgr2=0x7f99a80ace50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.136 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a80b1b60 0x7f99a80ace50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9998001eb0 tx=0x7f999800c8b0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.136 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 shutdown_connections 2022-11-27T16:03:58.137 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f9988066540 0x7f99880689f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.137 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f99a80b2640 0x7f99a80ad390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.137 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99a806d6c0 0x7f99a80c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.137 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 --2- 172.21.15.105:0/1523802251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f99a80b1b60 0x7f99a80ace50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.138 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 >> 172.21.15.105:0/1523802251 conn(0x7f99a80cc9d0 msgr2=0x7f99a80b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:58.138 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 shutdown_connections 2022-11-27T16:03:58.138 INFO:teuthology.misc.health.smithi105.stderr:2022-11-27T16:03:58.133+0000 7f99b01af700 1 -- 172.21.15.105:0/1523802251 wait complete. 2022-11-27T16:03:58.146 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-11-27T16:03:58.146 INFO:tasks.ceph:Creating RBD pool 2022-11-27T16:03:58.147 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T16:03:58.278 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.264+0000 7febdab0a700 1 Processor -- start 2022-11-27T16:03:58.278 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.264+0000 7febdab0a700 1 -- start start 2022-11-27T16:03:58.278 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febdab0a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7febd40b3ae0 0x7febd40b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:58.278 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febdab0a700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd415fdc0 0x7febd406f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:58.279 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febdab0a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40cc950 con 0x7febd40d1540 2022-11-27T16:03:58.279 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febdab0a700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40cc6b0 con 0x7febd40b3ae0 2022-11-27T16:03:58.279 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febdab0a700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40ccde0 con 0x7febd415fdc0 2022-11-27T16:03:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd88a6700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7febd40b3ae0 0x7febd40b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd90a7700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7febd40d1540 0x7febd406fef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45780/0 (socket says 172.21.15.105:45780) 2022-11-27T16:03:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd90a7700 1 -- 172.21.15.105:0/944579258 learned_addr learned my addr 172.21.15.105:0/944579258 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:03:58.280 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd3fff700 1 --2- 172.21.15.105:0/944579258 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd415fdc0 0x7febd406f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd88a6700 1 -- 172.21.15.105:0/944579258 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd415fdc0 msgr2=0x7febd406f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd88a6700 1 --2- 172.21.15.105:0/944579258 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd415fdc0 0x7febd406f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd88a6700 1 -- 172.21.15.105:0/944579258 >> v1:172.21.15.105:6789/0 conn(0x7febd40d1540 legacy=0x7febd406fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.281 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd88a6700 1 -- 172.21.15.105:0/944579258 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd41602c0 con 0x7febd40b3ae0 2022-11-27T16:03:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd88a6700 1 --2- 172.21.15.105:0/944579258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd40b3ae0 0x7febd40b3eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febc8009840 tx=0x7febc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f4703f5019a86c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.265+0000 7febd37fe700 1 -- 172.21.15.105:0/944579258 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc8004230 con 0x7febd40b3ae0 2022-11-27T16:03:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febd37fe700 1 -- 172.21.15.105:0/944579258 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc8004390 con 0x7febd40b3ae0 2022-11-27T16:03:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febd37fe700 1 -- 172.21.15.105:0/944579258 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80036a0 con 0x7febd40b3ae0 2022-11-27T16:03:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 -- 172.21.15.105:0/944579258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd40b3ae0 msgr2=0x7febd40b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 --2- 172.21.15.105:0/944579258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd40b3ae0 0x7febd40b3eb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7febc8009840 tx=0x7febc8009b50 comp rx=0 tx=0).stop 2022-11-27T16:03:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 -- 172.21.15.105:0/944579258 shutdown_connections 2022-11-27T16:03:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 --2- 172.21.15.105:0/944579258 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd40b3ae0 0x7febd40b3eb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 --2- 172.21.15.105:0/944579258 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd415fdc0 0x7febd406f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 -- 172.21.15.105:0/944579258 >> 172.21.15.105:0/944579258 conn(0x7febd40ceae0 msgr2=0x7febd40ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 -- 172.21.15.105:0/944579258 shutdown_connections 2022-11-27T16:03:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.266+0000 7febdab0a700 1 -- 172.21.15.105:0/944579258 wait complete. 2022-11-27T16:03:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.267+0000 7febdab0a700 1 Processor -- start 2022-11-27T16:03:58.285 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.267+0000 7febdab0a700 1 -- start start 2022-11-27T16:03:58.285 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febdab0a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febd40b3ae0 0x7febd4148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:58.285 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febdab0a700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd40d1540 0x7febd415de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:58.285 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febdab0a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd415fdc0 0x7febd415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febdab0a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40cae50 con 0x7febd415fdc0 2022-11-27T16:03:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febdab0a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41e5370 con 0x7febd40b3ae0 2022-11-27T16:03:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febdab0a700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd41e5650 con 0x7febd40d1540 2022-11-27T16:03:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd415fdc0 0x7febd415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd415fdc0 0x7febd415e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50912/0 (socket says 172.21.15.105:50912) 2022-11-27T16:03:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 -- 172.21.15.105:0/154889373 learned_addr learned my addr 172.21.15.105:0/154889373 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd3fff700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd40d1540 0x7febd415de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd88a6700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febd40b3ae0 0x7febd4148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 -- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd40d1540 msgr2=0x7febd415de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd40d1540 0x7febd415de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 -- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febd40b3ae0 msgr2=0x7febd4148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febd40b3ae0 0x7febd4148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.268+0000 7febd90a7700 1 -- 172.21.15.105:0/154889373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80094a0 con 0x7febd415fdc0 2022-11-27T16:03:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.269+0000 7febd90a7700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd415fdc0 0x7febd415e3c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7febcc00cca0 tx=0x7febcc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.269+0000 7febd17fa700 1 -- 172.21.15.105:0/154889373 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00ab30 con 0x7febd415fdc0 2022-11-27T16:03:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.269+0000 7febd17fa700 1 -- 172.21.15.105:0/154889373 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00ac90 con 0x7febd415fdc0 2022-11-27T16:03:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.269+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd41e5930 con 0x7febd415fdc0 2022-11-27T16:03:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.269+0000 7febd17fa700 1 -- 172.21.15.105:0/154889373 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc0070d0 con 0x7febd415fdc0 2022-11-27T16:03:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.269+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd4148980 con 0x7febd415fdc0 2022-11-27T16:03:58.290 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.270+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd41e5e50 con 0x7febd415fdc0 2022-11-27T16:03:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.270+0000 7febd17fa700 1 -- 172.21.15.105:0/154889373 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7febcc01a030 con 0x7febd415fdc0 2022-11-27T16:03:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.270+0000 7febd17fa700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7febbc066540 0x7febbc0689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.270+0000 7febd17fa700 1 -- 172.21.15.105:0/154889373 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4467+0+0 (secure 0 0 0) 0x7febcc082d30 con 0x7febd415fdc0 2022-11-27T16:03:58.291 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.270+0000 7febd88a6700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7febbc066540 0x7febbc0689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.271+0000 7febd88a6700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7febbc066540 0x7febbc0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febc8009840 tx=0x7febc8008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:58.292 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.275+0000 7febd17fa700 1 -- 172.21.15.105:0/154889373 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7febd41e5e50 con 0x7febd415fdc0 2022-11-27T16:03:58.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.452+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7febd41e5e50 con 0x7febd415fdc0 2022-11-27T16:03:58.901 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.899+0000 7febd17fa700 1 -- 172.21.15.105:0/154889373 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v14) v1 ==== 109+0+0 (secure 0 0 0) 0x7febd41e5e50 con 0x7febd415fdc0 2022-11-27T16:03:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7febbc066540 msgr2=0x7febbc0689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7febbc066540 0x7febbc0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7febc8009840 tx=0x7febc8008030 comp rx=0 tx=0).stop 2022-11-27T16:03:58.904 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd415fdc0 msgr2=0x7febd415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:58.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd415fdc0 0x7febd415e3c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7febcc00cca0 tx=0x7febcc00a060 comp rx=0 tx=0).stop 2022-11-27T16:03:58.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 shutdown_connections 2022-11-27T16:03:58.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7febbc066540 0x7febbc0689f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7febd415fdc0 0x7febd415e3c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.905 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7febd40b3ae0 0x7febd4148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 --2- 172.21.15.105:0/154889373 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7febd40d1540 0x7febd415de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 >> 172.21.15.105:0/154889373 conn(0x7febd40ceae0 msgr2=0x7febd40b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 shutdown_connections 2022-11-27T16:03:58.906 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:58.901+0000 7febdab0a700 1 -- 172.21.15.105:0/154889373 wait complete. 2022-11-27T16:03:58.907 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'rbd' created 2022-11-27T16:03:58.917 DEBUG:teuthology.orchestra.run.smithi105:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T16:03:59.044 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.042+0000 7f4df5cd7700 1 Processor -- start 2022-11-27T16:03:59.045 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.042+0000 7f4df5cd7700 1 -- start start 2022-11-27T16:03:59.053 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4df5cd7700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f4df00cc780 0x7f4df00b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4df5cd7700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 0x7f4df015f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.054 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4df5cd7700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df015fe70 con 0x7f4df015ba10 2022-11-27T16:03:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4df5cd7700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df015bde0 con 0x7f4df00cc780 2022-11-27T16:03:59.055 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4df5cd7700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00ccc10 con 0x7f4df015f4d0 2022-11-27T16:03:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4def7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4df015ba10 0x7f4df00d28e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45792/0 (socket says 172.21.15.105:45792) 2022-11-27T16:03:59.056 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4def7fe700 1 -- 172.21.15.105:0/2211688501 learned_addr learned my addr 172.21.15.105:0/2211688501 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:03:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4deffff700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 0x7f4df015f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323541350 0 0) 0x7f4df015fe70 con 0x7f4df015ba10 2022-11-27T16:03:59.057 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd00035b0 con 0x7f4df015ba10 2022-11-27T16:03:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.043+0000 7f4deeffd700 1 --2- 172.21.15.105:0/2211688501 >> v2:172.21.15.170:3300/0 conn(0x7f4df00cc780 0x7f4df00b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.058 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 179381839 0 0) 0x7f4dd00035b0 con 0x7f4df015ba10 2022-11-27T16:03:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 msgr2=0x7f4df015f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:59.059 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 0x7f4df015f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 >> v2:172.21.15.170:3300/0 conn(0x7f4df00cc780 msgr2=0x7f4df00b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:59.060 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 --2- 172.21.15.105:0/2211688501 >> v2:172.21.15.170:3300/0 conn(0x7f4df00cc780 0x7f4df00b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00cfda0 con 0x7f4df015ba10 2022-11-27T16:03:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3018764546 0 0) 0x7f4dd8001670 con 0x7f4df015ba10 2022-11-27T16:03:59.061 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 >> v1:172.21.15.105:6789/0 conn(0x7f4df015ba10 legacy=0x7f4df00d28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 0x7f4dd0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.062 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4dd0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.063 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd0004d90 0x7f4dd0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.063 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df015bde0 con 0x7f4dd0003f80 2022-11-27T16:03:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00ccc10 con 0x7f4dd0004d90 2022-11-27T16:03:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd000ba10 con 0x7f4df015f4d0 2022-11-27T16:03:59.064 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deeffd700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4dd0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4def7fe700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 0x7f4dd0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.065 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deffff700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd0004d90 0x7f4dd0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deeffd700 1 -- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 msgr2=0x7f4dd0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deeffd700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df015f4d0 0x7f4dd0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.066 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deeffd700 1 -- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd0004d90 msgr2=0x7f4dd0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:03:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deeffd700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4dd0004d90 0x7f4dd0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.067 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deeffd700 1 -- 172.21.15.105:0/2211688501 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd00035b0 con 0x7f4dd0003f80 2022-11-27T16:03:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4deeffd700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4dd0004850 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009b60 tx=0x7f4dd0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a1131b5b5908c2 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000e6c0 con 0x7f4dd0003f80 2022-11-27T16:03:59.068 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de000e820 con 0x7f4dd0003f80 2022-11-27T16:03:59.069 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.044+0000 7f4dee7fc700 1 -- 172.21.15.105:0/2211688501 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de000eb90 con 0x7f4dd0003f80 2022-11-27T16:03:59.069 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.045+0000 7f4df5cd7700 1 -- 172.21.15.105:0/2211688501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 msgr2=0x7f4dd0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:59.070 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.045+0000 7f4df5cd7700 1 --2- 172.21.15.105:0/2211688501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4dd0004850 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4de0009b60 tx=0x7f4dd0005880 comp rx=0 tx=0).stop 2022-11-27T16:03:59.070 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.045+0000 7f4deffff700 1 -- 172.21.15.105:0/2211688501 reap_dead start 2022-11-27T16:03:59.070 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.045+0000 7f4df5cd7700 1 -- 172.21.15.105:0/2211688501 shutdown_connections 2022-11-27T16:03:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.045+0000 7f4df5cd7700 1 -- 172.21.15.105:0/2211688501 >> 172.21.15.105:0/2211688501 conn(0x7f4df006f9b0 msgr2=0x7f4df006fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.045+0000 7f4df5cd7700 1 -- 172.21.15.105:0/2211688501 shutdown_connections 2022-11-27T16:03:59.071 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.045+0000 7f4df5cd7700 1 -- 172.21.15.105:0/2211688501 wait complete. 2022-11-27T16:03:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.046+0000 7f4df5cd7700 1 Processor -- start 2022-11-27T16:03:59.072 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.046+0000 7f4df5cd7700 1 -- start start 2022-11-27T16:03:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4df5cd7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4df00b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4df5cd7700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df00b4f60 0x7f4df01f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.073 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4df5cd7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4df00cbe70 0x7f4df00c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4df5cd7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00b60e0 con 0x7f4dd0003f80 2022-11-27T16:03:59.074 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4df5cd7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df01f11a0 con 0x7f4df00cbe70 2022-11-27T16:03:59.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4df5cd7700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00b5460 con 0x7f4df00b4f60 2022-11-27T16:03:59.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4df00b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4df00b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50938/0 (socket says 172.21.15.105:50938) 2022-11-27T16:03:59.076 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 -- 172.21.15.105:0/1845811889 learned_addr learned my addr 172.21.15.105:0/1845811889 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:03:59.076 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4deeffd700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df00b4f60 0x7f4df01f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 -- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df00b4f60 msgr2=0x7f4df01f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:59.077 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4deffff700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4df00cbe70 0x7f4df00c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df00b4f60 0x7f4df01f0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 -- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4df00cbe70 msgr2=0x7f4df00c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:59.078 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4df00cbe70 0x7f4df00c9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.079 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 -- 172.21.15.105:0/1845811889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00b5740 con 0x7f4dd0003f80 2022-11-27T16:03:59.079 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.047+0000 7f4def7fe700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4df00b6500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8000c00 tx=0x7f4dd80075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:59.080 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.048+0000 7f4de7fff700 1 -- 172.21.15.105:0/1845811889 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80077d0 con 0x7f4dd0003f80 2022-11-27T16:03:59.080 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.048+0000 7f4de7fff700 1 -- 172.21.15.105:0/1845811889 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd800f8d0 con 0x7f4dd0003f80 2022-11-27T16:03:59.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.048+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df00b5a20 con 0x7f4dd0003f80 2022-11-27T16:03:59.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.048+0000 7f4de7fff700 1 -- 172.21.15.105:0/1845811889 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800fc80 con 0x7f4dd0003f80 2022-11-27T16:03:59.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.048+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df00cc310 con 0x7f4dd0003f80 2022-11-27T16:03:59.082 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.049+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df00b6220 con 0x7f4dd0003f80 2022-11-27T16:03:59.082 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.049+0000 7f4de7fff700 1 -- 172.21.15.105:0/1845811889 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7f4dd801d460 con 0x7f4dd0003f80 2022-11-27T16:03:59.083 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.049+0000 7f4de7fff700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f4dd4066540 0x7f4dd40689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:03:59.083 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.049+0000 7f4de7fff700 1 -- 172.21.15.105:0/1845811889 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4809+0+0 (secure 0 0 0) 0x7f4dd8083430 con 0x7f4dd0003f80 2022-11-27T16:03:59.084 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.050+0000 7f4deeffd700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f4dd4066540 0x7f4dd40689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:03:59.084 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.050+0000 7f4deeffd700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f4dd4066540 0x7f4dd40689f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4df00b5390 tx=0x7f4de0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:03:59.085 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.054+0000 7f4de7fff700 1 -- 172.21.15.105:0/1845811889 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f4df00b6220 con 0x7f4dd0003f80 2022-11-27T16:03:59.232 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.230+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4df00b6220 con 0x7f4dd0003f80 2022-11-27T16:03:59.914 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.912+0000 7f4de7fff700 1 -- 172.21.15.105:0/1845811889 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v15) v1 ==== 173+0+0 (secure 0 0 0) 0x7f4df00b6220 con 0x7f4dd0003f80 2022-11-27T16:03:59.917 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.914+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f4dd4066540 msgr2=0x7f4dd40689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.914+0000 7f4df5cd7700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f4dd4066540 0x7f4dd40689f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4df00b5390 tx=0x7f4de0008030 comp rx=0 tx=0).stop 2022-11-27T16:03:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.914+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 msgr2=0x7f4df00b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:03:59.918 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.914+0000 7f4df5cd7700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4df00b6500 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8000c00 tx=0x7f4dd80075f0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.919 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 shutdown_connections 2022-11-27T16:03:59.919 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f4dd4066540 0x7f4dd40689f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4df00cbe70 0x7f4df00c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.920 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4dd0003f80 0x7f4df00b6500 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.921 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 --2- 172.21.15.105:0/1845811889 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f4df00b4f60 0x7f4df01f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:03:59.921 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 >> 172.21.15.105:0/1845811889 conn(0x7f4df006f9b0 msgr2=0x7f4df00ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:03:59.921 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 shutdown_connections 2022-11-27T16:03:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:03:59.915+0000 7f4df5cd7700 1 -- 172.21.15.105:0/1845811889 wait complete. 2022-11-27T16:03:59.922 INFO:teuthology.orchestra.run.smithi105.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T16:03:59.929 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T16:03:59.930 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T16:03:59.930 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T16:03:59.933 INFO:tasks.ceph.mds.a:Started 2022-11-27T16:03:59.934 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T16:03:59.934 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T16:03:59.936 INFO:tasks.ceph.mds.c:Started 2022-11-27T16:03:59.936 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T16:03:59.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T16:03:59.940 INFO:tasks.ceph.mds.b:Started 2022-11-27T16:03:59.940 INFO:tasks.ceph.mds.d:Restarting daemon 2022-11-27T16:03:59.941 DEBUG:teuthology.orchestra.run.smithi170:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-11-27T16:03:59.943 INFO:tasks.ceph.mds.d:Started 2022-11-27T16:03:59.944 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T16:03:59.944 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:04:00.011 INFO:tasks.ceph.mds.a.smithi105.stdout:starting mds.a at 2022-11-27T16:04:00.013 INFO:tasks.ceph.mds.c.smithi105.stdout:starting mds.c at 2022-11-27T16:04:00.027 INFO:tasks.ceph.mds.d.smithi170.stdout:starting mds.d at 2022-11-27T16:04:00.027 INFO:tasks.ceph.mds.b.smithi170.stdout:starting mds.b at 2022-11-27T16:04:00.074 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.072+0000 7fc169f06700 1 Processor -- start 2022-11-27T16:04:00.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.072+0000 7fc169f06700 1 -- start start 2022-11-27T16:04:00.080 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc169f06700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1640cd350 0x7fc1640cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc169f06700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc16415c100 0x7fc164069390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc169f06700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1640cdc60 con 0x7fc1640c44f0 2022-11-27T16:04:00.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc169f06700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1640c9e00 con 0x7fc1640cd350 2022-11-27T16:04:00.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc169f06700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1640ca0e0 con 0x7fc16415c100 2022-11-27T16:04:00.082 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc1637fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1640cd350 0x7fc1640cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.082 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc1637fe700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fc1640cd350 0x7fc1640cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:50978/0 (socket says 172.21.15.105:50978) 2022-11-27T16:04:00.082 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc1637fe700 1 -- 172.21.15.105:0/3371947943 learned_addr learned my addr 172.21.15.105:0/3371947943 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:00.082 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc163fff700 1 --2- 172.21.15.105:0/3371947943 >> v2:172.21.15.170:3300/0 conn(0x7fc16415c100 0x7fc164069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc1637fe700 1 -- 172.21.15.105:0/3371947943 >> v2:172.21.15.170:3300/0 conn(0x7fc16415c100 msgr2=0x7fc164069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc1637fe700 1 --2- 172.21.15.105:0/3371947943 >> v2:172.21.15.170:3300/0 conn(0x7fc16415c100 0x7fc164069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc1637fe700 1 -- 172.21.15.105:0/3371947943 >> v1:172.21.15.170:6789/0 conn(0x7fc1640c44f0 legacy=0x7fc1640caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.083 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.073+0000 7fc1637fe700 1 -- 172.21.15.105:0/3371947943 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1640ca3c0 con 0x7fc1640cd350 2022-11-27T16:04:00.084 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc1637fe700 1 --2- 172.21.15.105:0/3371947943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc1640cd720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009e50 tx=0x7fc15c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a43b4451cd1a29b5 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.084 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc1627fc700 1 -- 172.21.15.105:0/3371947943 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00e070 con 0x7fc1640cd350 2022-11-27T16:04:00.084 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc1627fc700 1 -- 172.21.15.105:0/3371947943 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15c00b980 con 0x7fc1640cd350 2022-11-27T16:04:00.084 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc1627fc700 1 -- 172.21.15.105:0/3371947943 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15c00bd30 con 0x7fc1640cd350 2022-11-27T16:04:00.085 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 -- 172.21.15.105:0/3371947943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 msgr2=0x7fc1640cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.085 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 --2- 172.21.15.105:0/3371947943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc1640cd720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009e50 tx=0x7fc15c00b330 comp rx=0 tx=0).stop 2022-11-27T16:04:00.085 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 -- 172.21.15.105:0/3371947943 shutdown_connections 2022-11-27T16:04:00.085 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 --2- 172.21.15.105:0/3371947943 >> v2:172.21.15.170:3300/0 conn(0x7fc16415c100 0x7fc164069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.085 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 --2- 172.21.15.105:0/3371947943 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc1640cd720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.086 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 -- 172.21.15.105:0/3371947943 >> 172.21.15.105:0/3371947943 conn(0x7fc1640cb7f0 msgr2=0x7fc1640cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:00.086 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 -- 172.21.15.105:0/3371947943 shutdown_connections 2022-11-27T16:04:00.086 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.074+0000 7fc169f06700 1 -- 172.21.15.105:0/3371947943 wait complete. 2022-11-27T16:04:00.086 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.075+0000 7fc169f06700 1 Processor -- start 2022-11-27T16:04:00.087 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.076+0000 7fc169f06700 1 -- start start 2022-11-27T16:04:00.087 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.076+0000 7fc169f06700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1640c44f0 0x7fc1640c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.087 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.076+0000 7fc169f06700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc164149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.087 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.076+0000 7fc169f06700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc16415c100 0x7fc16406c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.088 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.076+0000 7fc169f06700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc164157d20 con 0x7fc1640cd350 2022-11-27T16:04:00.088 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.076+0000 7fc169f06700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640c6e30 con 0x7fc1640c44f0 2022-11-27T16:04:00.088 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.076+0000 7fc169f06700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1640c6220 con 0x7fc16415c100 2022-11-27T16:04:00.088 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc164149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.088 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc164149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51922/0 (socket says 172.21.15.105:51922) 2022-11-27T16:04:00.089 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 -- 172.21.15.105:0/2828064223 learned_addr learned my addr 172.21.15.105:0/2828064223 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:00.089 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc1637fe700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1640c44f0 0x7fc1640c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.089 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc163fff700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc16415c100 0x7fc16406c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.089 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 -- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc16415c100 msgr2=0x7fc16406c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc16415c100 0x7fc16406c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 -- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1640c44f0 msgr2=0x7fc1640c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1640c44f0 0x7fc1640c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 -- 172.21.15.105:0/2828064223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c009b30 con 0x7fc1640cd350 2022-11-27T16:04:00.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc163fff700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc16415c100 0x7fc16406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:04:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc162ffd700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc164149e90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00c3c0 tx=0x7fc14c00c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc15bfff700 1 -- 172.21.15.105:0/2828064223 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c0052d0 con 0x7fc1640cd350 2022-11-27T16:04:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc15bfff700 1 -- 172.21.15.105:0/2828064223 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14c005430 con 0x7fc1640cd350 2022-11-27T16:04:00.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1640c6500 con 0x7fc1640cd350 2022-11-27T16:04:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc15bfff700 1 -- 172.21.15.105:0/2828064223 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14c006aa0 con 0x7fc1640cd350 2022-11-27T16:04:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.077+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1640c6ac0 con 0x7fc1640cd350 2022-11-27T16:04:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.078+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1640c6c50 con 0x7fc1640cd350 2022-11-27T16:04:00.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.079+0000 7fc15bfff700 1 -- 172.21.15.105:0/2828064223 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fc14c010050 con 0x7fc1640cd350 2022-11-27T16:04:00.093 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.079+0000 7fc15bfff700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc148066540 0x7fc1480689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.079+0000 7fc15bfff700 1 -- 172.21.15.105:0/2828064223 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fc14c082410 con 0x7fc1640cd350 2022-11-27T16:04:00.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.079+0000 7fc1637fe700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc148066540 0x7fc1480689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.079+0000 7fc1637fe700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc148066540 0x7fc1480689f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00b370 tx=0x7fc15c009000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.083+0000 7fc15bfff700 1 -- 172.21.15.105:0/2828064223 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fc14c049bd0 con 0x7fc1640cd350 2022-11-27T16:04:00.262 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.260+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc164067420 con 0x7fc1640cd350 2022-11-27T16:04:00.264 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.261+0000 7fc15bfff700 1 -- 172.21.15.105:0/2828064223 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+13808 (secure 0 0 0) 0x7fc14c04fe70 con 0x7fc1640cd350 2022-11-27T16:04:00.265 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:04:00.265 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":15,"fsid":"558b439a-8d62-4391-a123-f6ff0625ded9","created":"2022-11-27T16:03:46.292863+0000","modified":"2022-11-27T16:03:59.898954+0000","last_up_change":"2022-11-27T16:03:57.895226+0000","last_in_change":"2022-11-27T16:03:49.700155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2022-11-27T16:03:58.470316+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2022-11-27T16:03:59.588131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"a877ed0a-6953-4642-9d56-f1eae305b6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":112238},{"type":"v1","addr":"172.21.15.105:6801","nonce":112238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":112238},{"type":"v1","addr":"172.21.15.105:6803","nonce":112238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":112238},{"type":"v1","addr":"172.21.15.105:6807","nonce":112238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":112238},{"type":"v1","addr":"172.21.15.105:6805","nonce":112238}]},"public_addr":"172.21.15.105:6801/112238","cluster_addr":"172.21.15.105:6803/112238","heartbeat_back_addr":"172.21.15.105:6807/112238","heartbeat_front_addr":"172.21.15.105:6805/112238","state":["exists","up"]},{"osd":1,"uuid":"dae5370d-59b3-458c-9821-f88f47769ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":112239},{"type":"v1","addr":"172.21.15.105:6809","nonce":112239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":112239},{"type":"v1","addr":"172.21.15.105:6811","nonce":112239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":112239},{"type":"v1","addr":"172.21.15.105:6815","nonce":112239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":112239},{"type":"v1","addr":"172.21.15.105:6813","nonce":112239}]},"public_addr":"172.21.15.105:6809/112239","cluster_addr":"172.21.15.105:6811/112239","heartbeat_back_addr":"172.21.15.105:6815/112239","heartbeat_front_addr":"172.21.15.105:6813/112239","state":["exists","up"]},{"osd":2,"uuid":"e34db7ee-5ac2-4682-8b3d-77781aff8feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":112240},{"type":"v1","addr":"172.21.15.105:6817","nonce":112240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":112240},{"type":"v1","addr":"172.21.15.105:6819","nonce":112240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":112240},{"type":"v1","addr":"172.21.15.105:6823","nonce":112240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":112240},{"type":"v1","addr":"172.21.15.105:6821","nonce":112240}]},"public_addr":"172.21.15.105:6817/112240","cluster_addr":"172.21.15.105:6819/112240","heartbeat_back_addr":"172.21.15.105:6823/112240","heartbeat_front_addr":"172.21.15.105:6821/112240","state":["exists","up"]},{"osd":3,"uuid":"98f2fda2-a32e-4e5c-ad39-c3d67013ccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":112241},{"type":"v1","addr":"172.21.15.105:6825","nonce":112241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":112241},{"type":"v1","addr":"172.21.15.105:6827","nonce":112241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":112241},{"type":"v1","addr":"172.21.15.105:6831","nonce":112241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":112241},{"type":"v1","addr":"172.21.15.105:6829","nonce":112241}]},"public_addr":"172.21.15.105:6825/112241","cluster_addr":"172.21.15.105:6827/112241","heartbeat_back_addr":"172.21.15.105:6831/112241","heartbeat_front_addr":"172.21.15.105:6829/112241","state":["exists","up"]},{"osd":4,"uuid":"ec8582a4-93d8-4e63-8b94-5386bf561c11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":112474},{"type":"v1","addr":"172.21.15.170:6801","nonce":112474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":112474},{"type":"v1","addr":"172.21.15.170:6803","nonce":112474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":112474},{"type":"v1","addr":"172.21.15.170:6807","nonce":112474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":112474},{"type":"v1","addr":"172.21.15.170:6805","nonce":112474}]},"public_addr":"172.21.15.170:6801/112474","cluster_addr":"172.21.15.170:6803/112474","heartbeat_back_addr":"172.21.15.170:6807/112474","heartbeat_front_addr":"172.21.15.170:6805/112474","state":["exists","up"]},{"osd":5,"uuid":"55b1b3a8-5dac-4f1e-b0d6-1871422ae097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":112486},{"type":"v1","addr":"172.21.15.170:6818","nonce":112486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6819","nonce":112486},{"type":"v1","addr":"172.21.15.170:6821","nonce":112486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":112486},{"type":"v1","addr":"172.21.15.170:6827","nonce":112486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":112486},{"type":"v1","addr":"172.21.15.170:6824","nonce":112486}]},"public_addr":"172.21.15.170:6818/112486","cluster_addr":"172.21.15.170:6821/112486","heartbeat_back_addr":"172.21.15.170:6827/112486","heartbeat_front_addr":"172.21.15.170:6824/112486","state":["exists","up"]},{"osd":6,"uuid":"64ff9585-481c-4ab8-a276-dad128c5dba9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":112475},{"type":"v1","addr":"172.21.15.170:6809","nonce":112475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":112475},{"type":"v1","addr":"172.21.15.170:6811","nonce":112475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":112475},{"type":"v1","addr":"172.21.15.170:6815","nonce":112475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":112475},{"type":"v1","addr":"172.21.15.170:6813","nonce":112475}]},"public_addr":"172.21.15.170:6809/112475","cluster_addr":"172.21.15.170:6811/112475","heartbeat_back_addr":"172.21.15.170:6815/112475","heartbeat_front_addr":"172.21.15.170:6813/112475","state":["exists","up"]},{"osd":7,"uuid":"9a2592f7-a19d-49df-beb3-004cd10cf16b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":112495},{"type":"v1","addr":"172.21.15.170:6820","nonce":112495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":112495},{"type":"v1","addr":"172.21.15.170:6825","nonce":112495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":112495},{"type":"v1","addr":"172.21.15.170:6831","nonce":112495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":112495},{"type":"v1","addr":"172.21.15.170:6829","nonce":112495}]},"public_addr":"172.21.15.170:6820/112495","cluster_addr":"172.21.15.170:6825/112495","heartbeat_back_addr":"172.21.15.170:6831/112495","heartbeat_front_addr":"172.21.15.170:6829/112495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.603339+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.577029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.764972+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.940421+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.831837+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.894911+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.805503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.855604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:04:00.267 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc148066540 msgr2=0x7fc1480689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.268 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc148066540 0x7fc1480689f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc15c00b370 tx=0x7fc15c009000 comp rx=0 tx=0).stop 2022-11-27T16:04:00.268 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 msgr2=0x7fc164149e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.268 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc164149e90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00c3c0 tx=0x7fc14c00c6d0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.269 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 shutdown_connections 2022-11-27T16:04:00.269 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc148066540 0x7fc1480689f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.270 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc16415c100 0x7fc16406c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.270 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1640c44f0 0x7fc1640c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.270 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 --2- 172.21.15.105:0/2828064223 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1640cd350 0x7fc164149e90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.271 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 >> 172.21.15.105:0/2828064223 conn(0x7fc1640cb7f0 msgr2=0x7fc1640cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:00.271 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 shutdown_connections 2022-11-27T16:04:00.271 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.263+0000 7fc169f06700 1 -- 172.21.15.105:0/2828064223 wait complete. 2022-11-27T16:04:00.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:03:58.470316+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:03:59.588131+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}}] 2022-11-27T16:04:00.278 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:04:00.400 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.398+0000 7fd10711a700 1 Processor -- start 2022-11-27T16:04:00.401 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.398+0000 7fd10711a700 1 -- start start 2022-11-27T16:04:00.407 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd10711a700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd1000cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.407 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd10711a700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1000b18d0 0x7fd10006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.407 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd10711a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1000b1d10 con 0x7fd1000cc0e0 2022-11-27T16:04:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd10711a700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1000b2800 con 0x7fd1000b18d0 2022-11-27T16:04:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd10711a700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd10006ef00 con 0x7fd1000ccc80 2022-11-27T16:04:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd1056b7700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fd1000b18d0 0x7fd10006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.408 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd0fffff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fd1000cc0e0 0x7fd10015fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43558/0 (socket says 172.21.15.105:43558) 2022-11-27T16:04:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd0fffff700 1 -- 172.21.15.105:0/865502675 learned_addr learned my addr 172.21.15.105:0/865502675 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:04:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd104eb6700 1 --2- 172.21.15.105:0/865502675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd1000cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd1056b7700 1 -- 172.21.15.105:0/865502675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 msgr2=0x7fd1000cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd1056b7700 1 --2- 172.21.15.105:0/865502675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd1000cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.409 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd1056b7700 1 -- 172.21.15.105:0/865502675 >> v1:172.21.15.105:6789/0 conn(0x7fd1000cc0e0 legacy=0x7fd10015fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd1056b7700 1 -- 172.21.15.105:0/865502675 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000b1f60 con 0x7fd1000b18d0 2022-11-27T16:04:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd0ff7fe700 1 -- 172.21.15.105:0/865502675 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079328065 0 0) 0x7fd1000b1d10 con 0x7fd1000cc0e0 2022-11-27T16:04:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd104eb6700 1 --2- 172.21.15.105:0/865502675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd1000cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:04:00.410 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.399+0000 7fd1056b7700 1 --2- 172.21.15.105:0/865502675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd10006a5b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009d90 tx=0x7fd0f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9a78553a3e0d6d server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd0ff7fe700 1 -- 172.21.15.105:0/865502675 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800e070 con 0x7fd1000b18d0 2022-11-27T16:04:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd0ff7fe700 1 -- 172.21.15.105:0/865502675 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f800bbd0 con 0x7fd1000b18d0 2022-11-27T16:04:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd0ff7fe700 1 -- 172.21.15.105:0/865502675 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8012430 con 0x7fd1000b18d0 2022-11-27T16:04:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 -- 172.21.15.105:0/865502675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 msgr2=0x7fd10006a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.411 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 --2- 172.21.15.105:0/865502675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd10006a5b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8009d90 tx=0x7fd0f800b330 comp rx=0 tx=0).stop 2022-11-27T16:04:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 -- 172.21.15.105:0/865502675 shutdown_connections 2022-11-27T16:04:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 --2- 172.21.15.105:0/865502675 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd1000cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 --2- 172.21.15.105:0/865502675 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd10006a5b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.412 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 -- 172.21.15.105:0/865502675 >> 172.21.15.105:0/865502675 conn(0x7fd10006cfe0 msgr2=0x7fd10006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:00.413 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 -- 172.21.15.105:0/865502675 shutdown_connections 2022-11-27T16:04:00.413 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.400+0000 7fd10711a700 1 -- 172.21.15.105:0/865502675 wait complete. 2022-11-27T16:04:00.413 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.401+0000 7fd10711a700 1 Processor -- start 2022-11-27T16:04:00.413 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.401+0000 7fd10711a700 1 -- start start 2022-11-27T16:04:00.413 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd10711a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd100147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.414 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd10711a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1000cc0e0 0x7fd100147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.414 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd10711a700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd10015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.414 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd10711a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10015b410 con 0x7fd1000b18d0 2022-11-27T16:04:00.414 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd10711a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10015c130 con 0x7fd1000cc0e0 2022-11-27T16:04:00.415 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd10711a700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd100147fc0 con 0x7fd1000ccc80 2022-11-27T16:04:00.415 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd104eb6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd100147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.415 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd104eb6700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd100147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51938/0 (socket says 172.21.15.105:51938) 2022-11-27T16:04:00.415 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd104eb6700 1 -- 172.21.15.105:0/199345513 learned_addr learned my addr 172.21.15.105:0/199345513 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:00.416 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd1056b7700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd10015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.416 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.402+0000 7fd0fffff700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1000cc0e0 0x7fd100147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.416 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd104eb6700 1 -- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 msgr2=0x7fd10015e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.416 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd104eb6700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd10015e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.416 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd104eb6700 1 -- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1000cc0e0 msgr2=0x7fd100147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd104eb6700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1000cc0e0 0x7fd100147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd104eb6700 1 -- 172.21.15.105:0/199345513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8009a70 con 0x7fd1000b18d0 2022-11-27T16:04:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd104eb6700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd100147a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c750 tx=0x7fd0f400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.417 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd0fd7fa700 1 -- 172.21.15.105:0/199345513 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011800 con 0x7fd1000b18d0 2022-11-27T16:04:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd10015c340 con 0x7fd1000b18d0 2022-11-27T16:04:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd0fd7fa700 1 -- 172.21.15.105:0/199345513 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4011960 con 0x7fd1000b18d0 2022-11-27T16:04:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd0fd7fa700 1 -- 172.21.15.105:0/199345513 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011cd0 con 0x7fd1000b18d0 2022-11-27T16:04:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.403+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd10015c900 con 0x7fd1000b18d0 2022-11-27T16:04:00.418 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.404+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd100067420 con 0x7fd1000b18d0 2022-11-27T16:04:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.404+0000 7fd0fd7fa700 1 -- 172.21.15.105:0/199345513 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fd0f4020070 con 0x7fd1000b18d0 2022-11-27T16:04:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.405+0000 7fd0fd7fa700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fd0e8066540 0x7fd0e80689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.405+0000 7fd0fffff700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fd0e8066540 0x7fd0e80689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.419 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.405+0000 7fd0fd7fa700 1 -- 172.21.15.105:0/199345513 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fd0f4083260 con 0x7fd1000b18d0 2022-11-27T16:04:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.405+0000 7fd0fffff700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fd0e8066540 0x7fd0e80689f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd10015b9e0 tx=0x7fd0ec007e70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.420 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.409+0000 7fd0fd7fa700 1 -- 172.21.15.105:0/199345513 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fd0f4050bb0 con 0x7fd1000b18d0 2022-11-27T16:04:00.585 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.583+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd1000b2240 con 0x7fd1000b18d0 2022-11-27T16:04:00.586 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.584+0000 7fd0fd7fa700 1 -- 172.21.15.105:0/199345513 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd0f40541d0 con 0x7fd1000b18d0 2022-11-27T16:04:00.586 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 8 2022-11-27T16:04:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fd0e8066540 msgr2=0x7fd0e80689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.588 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fd0e8066540 0x7fd0e80689f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd10015b9e0 tx=0x7fd0ec007e70 comp rx=0 tx=0).stop 2022-11-27T16:04:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 msgr2=0x7fd100147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd100147a80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd0f400c750 tx=0x7fd0f400ca60 comp rx=0 tx=0).stop 2022-11-27T16:04:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 shutdown_connections 2022-11-27T16:04:00.589 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fd0e8066540 0x7fd0e80689f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fd1000ccc80 0x7fd10015e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1000b18d0 0x7fd100147a80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 --2- 172.21.15.105:0/199345513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd1000cc0e0 0x7fd100147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.590 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 >> 172.21.15.105:0/199345513 conn(0x7fd10006cfe0 msgr2=0x7fd1000c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 shutdown_connections 2022-11-27T16:04:00.591 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.586+0000 7fd10711a700 1 -- 172.21.15.105:0/199345513 wait complete. 2022-11-27T16:04:00.600 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:04:00.723 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.721+0000 7fc14bc7e700 1 Processor -- start 2022-11-27T16:04:00.724 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.721+0000 7fc14bc7e700 1 -- start start 2022-11-27T16:04:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc14bc7e700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fc1440cd3b0 0x7fc14406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc14bc7e700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc144069510 con 0x7fc144158970 2022-11-27T16:04:00.731 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc14bc7e700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1440697f0 con 0x7fc14406aaf0 2022-11-27T16:04:00.732 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc14bc7e700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc144069ad0 con 0x7fc1440cd3b0 2022-11-27T16:04:00.732 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc149a1a700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc144158970 0x7fc14415cd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43566/0 (socket says 172.21.15.105:43566) 2022-11-27T16:04:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc149a1a700 1 -- 172.21.15.105:0/1766211746 learned_addr learned my addr 172.21.15.105:0/1766211746 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:04:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc149219700 1 --2- 172.21.15.105:0/1766211746 >> v2:172.21.15.170:3300/0 conn(0x7fc1440cd3b0 0x7fc14406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985122396 0 0) 0x7fc144069510 con 0x7fc144158970 2022-11-27T16:04:00.733 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc12c0035b0 con 0x7fc144158970 2022-11-27T16:04:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3840145687 0 0) 0x7fc12c0035b0 con 0x7fc144158970 2022-11-27T16:04:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 >> v2:172.21.15.170:3300/0 conn(0x7fc1440cd3b0 msgr2=0x7fc14406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 --2- 172.21.15.105:0/1766211746 >> v2:172.21.15.170:3300/0 conn(0x7fc1440cd3b0 0x7fc14406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.734 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 >> v1:172.21.15.170:6789/0 conn(0x7fc14406aaf0 legacy=0x7fc14406bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc144069db0 con 0x7fc144158970 2022-11-27T16:04:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3018764546 0 0) 0x7fc140001670 con 0x7fc144158970 2022-11-27T16:04:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 >> v1:172.21.15.105:6789/0 conn(0x7fc144158970 legacy=0x7fc14415cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.735 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc12c003a00 0x7fc12c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc12c004850 0x7fc12c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c005200 0x7fc12c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc144069ad0 con 0x7fc12c003a00 2022-11-27T16:04:00.736 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c00c190 con 0x7fc12c004850 2022-11-27T16:04:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c00c350 con 0x7fc12c005200 2022-11-27T16:04:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc149219700 1 --2- 172.21.15.105:0/1766211746 >> v2:172.21.15.170:3300/0 conn(0x7fc1440cd3b0 0x7fc14406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:04:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.722+0000 7fc149a1a700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc12c003a00 0x7fc12c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.737 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14a21b700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c005200 0x7fc12c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.738 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149219700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc12c004850 0x7fc12c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.738 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149a1a700 1 -- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c005200 msgr2=0x7fc12c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:00.738 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149a1a700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c005200 0x7fc12c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.738 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149a1a700 1 -- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc12c004850 msgr2=0x7fc12c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:00.738 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149a1a700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc12c004850 0x7fc12c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149a1a700 1 -- 172.21.15.105:0/1766211746 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0035b0 con 0x7fc12c003a00 2022-11-27T16:04:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149219700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc12c004850 0x7fc12c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:04:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14a21b700 1 -- 172.21.15.105:0/1766211746 reap_dead start 2022-11-27T16:04:00.739 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc149a1a700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc12c003a00 0x7fc12c003e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c210 tx=0x7fc12c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46059b431c1378ca server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140007c10 con 0x7fc12c003a00 2022-11-27T16:04:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc14000cdd0 con 0x7fc12c003a00 2022-11-27T16:04:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc148a18700 1 -- 172.21.15.105:0/1766211746 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140005ba0 con 0x7fc12c003a00 2022-11-27T16:04:00.740 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14bc7e700 1 -- 172.21.15.105:0/1766211746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc12c003a00 msgr2=0x7fc12c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc12c003a00 0x7fc12c003e10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc14000c210 tx=0x7fc12c005880 comp rx=0 tx=0).stop 2022-11-27T16:04:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14bc7e700 1 -- 172.21.15.105:0/1766211746 shutdown_connections 2022-11-27T16:04:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/1766211746 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc12c003a00 0x7fc12c003e10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14bc7e700 1 -- 172.21.15.105:0/1766211746 >> 172.21.15.105:0/1766211746 conn(0x7fc1440cb7e0 msgr2=0x7fc1440cbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:00.741 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.723+0000 7fc14bc7e700 1 -- 172.21.15.105:0/1766211746 shutdown_connections 2022-11-27T16:04:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.724+0000 7fc14bc7e700 1 -- 172.21.15.105:0/1766211746 wait complete. 2022-11-27T16:04:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 Processor -- start 2022-11-27T16:04:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 -- start start 2022-11-27T16:04:00.742 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c003a00 0x7fc1440afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1440c8ec0 0x7fc1441f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1440b1ee0 0x7fc1440c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440c8700 con 0x7fc1440c8ec0 2022-11-27T16:04:00.743 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440c5ed0 con 0x7fc1440b1ee0 2022-11-27T16:04:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc14bc7e700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1440c61b0 con 0x7fc12c003a00 2022-11-27T16:04:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc149219700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1440c8ec0 0x7fc1441f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc149219700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1440c8ec0 0x7fc1441f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51968/0 (socket says 172.21.15.105:51968) 2022-11-27T16:04:00.744 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.725+0000 7fc149219700 1 -- 172.21.15.105:0/239970419 learned_addr learned my addr 172.21.15.105:0/239970419 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc149a1a700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c003a00 0x7fc1440afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc14a21b700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1440b1ee0 0x7fc1440c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc149219700 1 -- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c003a00 msgr2=0x7fc1440afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc149219700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c003a00 0x7fc1440afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.745 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc149219700 1 -- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1440b1ee0 msgr2=0x7fc1440c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc149219700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1440b1ee0 0x7fc1440c7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc149219700 1 -- 172.21.15.105:0/239970419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1440c89a0 con 0x7fc1440c8ec0 2022-11-27T16:04:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc149219700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1440c8ec0 0x7fc1441f0ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc134010840 tx=0x7fc134010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.746 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc1327fc700 1 -- 172.21.15.105:0/239970419 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134015880 con 0x7fc1440c8ec0 2022-11-27T16:04:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc1327fc700 1 -- 172.21.15.105:0/239970419 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13400c050 con 0x7fc1440c8ec0 2022-11-27T16:04:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1440b2350 con 0x7fc1440c8ec0 2022-11-27T16:04:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc1327fc700 1 -- 172.21.15.105:0/239970419 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134015c20 con 0x7fc1440c8ec0 2022-11-27T16:04:00.747 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.726+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1440c7f70 con 0x7fc1440c8ec0 2022-11-27T16:04:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.727+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1440c8d10 con 0x7fc1440c8ec0 2022-11-27T16:04:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.727+0000 7fc1327fc700 1 -- 172.21.15.105:0/239970419 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fc134025070 con 0x7fc1440c8ec0 2022-11-27T16:04:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.728+0000 7fc1327fc700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc124066540 0x7fc1240689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.728+0000 7fc149a1a700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc124066540 0x7fc1240689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:00.748 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.728+0000 7fc1327fc700 1 -- 172.21.15.105:0/239970419 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fc134083720 con 0x7fc1440c8ec0 2022-11-27T16:04:00.749 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.728+0000 7fc149a1a700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc124066540 0x7fc1240689f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc140000c00 tx=0x7fc140007000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:00.749 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.732+0000 7fc1327fc700 1 -- 172.21.15.105:0/239970419 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fc1340511e0 con 0x7fc1440c8ec0 2022-11-27T16:04:00.912 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.910+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc144067420 con 0x7fc1440c8ec0 2022-11-27T16:04:00.914 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.911+0000 7fc1327fc700 1 -- 172.21.15.105:0/239970419 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc134054800 con 0x7fc1440c8ec0 2022-11-27T16:04:00.914 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2022-11-27T16:04:00.917 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc124066540 msgr2=0x7fc1240689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.917 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc124066540 0x7fc1240689f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc140000c00 tx=0x7fc140007000 comp rx=0 tx=0).stop 2022-11-27T16:04:00.917 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1440c8ec0 msgr2=0x7fc1441f0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:00.918 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1440c8ec0 0x7fc1441f0ea0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc134010840 tx=0x7fc134010b50 comp rx=0 tx=0).stop 2022-11-27T16:04:00.918 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 shutdown_connections 2022-11-27T16:04:00.918 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fc124066540 0x7fc1240689f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.918 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fc12c003a00 0x7fc1440afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1440b1ee0 0x7fc1440c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 --2- 172.21.15.105:0/239970419 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1440c8ec0 0x7fc1441f0ea0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 >> 172.21.15.105:0/239970419 conn(0x7fc1440cb7e0 msgr2=0x7fc14406fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 shutdown_connections 2022-11-27T16:04:00.919 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:00.913+0000 7fc14bc7e700 1 -- 172.21.15.105:0/239970419 wait complete. 2022-11-27T16:04:00.928 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T16:04:01.059 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.057+0000 7fac28ef3700 1 Processor -- start 2022-11-27T16:04:01.060 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.057+0000 7fac28ef3700 1 -- start start 2022-11-27T16:04:01.063 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac28ef3700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fac240b3ae0 0x7fac240b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.063 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac28ef3700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac2415fdc0 0x7fac2406f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.063 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac28ef3700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac240cc950 con 0x7fac240d1540 2022-11-27T16:04:01.063 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac28ef3700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240cc6b0 con 0x7fac240b3ae0 2022-11-27T16:04:01.064 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac28ef3700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240ccde0 con 0x7fac2415fdc0 2022-11-27T16:04:01.064 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac2259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fac240b3ae0 0x7fac240b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.064 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac2259c700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7fac240b3ae0 0x7fac240b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51996/0 (socket says 172.21.15.105:51996) 2022-11-27T16:04:01.064 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac2259c700 1 -- 172.21.15.105:0/4093078854 learned_addr learned my addr 172.21.15.105:0/4093078854 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac21d9b700 1 --2- 172.21.15.105:0/4093078854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac2415fdc0 0x7fac2406f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac2259c700 1 -- 172.21.15.105:0/4093078854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac2415fdc0 msgr2=0x7fac2406f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac2259c700 1 --2- 172.21.15.105:0/4093078854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac2415fdc0 0x7fac2406f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.065 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac2259c700 1 -- 172.21.15.105:0/4093078854 >> v1:172.21.15.170:6789/0 conn(0x7fac240d1540 legacy=0x7fac2406fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.058+0000 7fac2259c700 1 -- 172.21.15.105:0/4093078854 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac241602c0 con 0x7fac240b3ae0 2022-11-27T16:04:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac2259c700 1 --2- 172.21.15.105:0/4093078854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac240b3ae0 0x7fac240b3eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac180098f0 tx=0x7fac18009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638c212dfb7dcfa7 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac2159a700 1 -- 172.21.15.105:0/4093078854 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18011830 con 0x7fac240b3ae0 2022-11-27T16:04:01.066 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac2159a700 1 -- 172.21.15.105:0/4093078854 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac18011990 con 0x7fac240b3ae0 2022-11-27T16:04:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac2159a700 1 -- 172.21.15.105:0/4093078854 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac18011d20 con 0x7fac240b3ae0 2022-11-27T16:04:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 -- 172.21.15.105:0/4093078854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac240b3ae0 msgr2=0x7fac240b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 --2- 172.21.15.105:0/4093078854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac240b3ae0 0x7fac240b3eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fac180098f0 tx=0x7fac18009c00 comp rx=0 tx=0).stop 2022-11-27T16:04:01.067 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 -- 172.21.15.105:0/4093078854 shutdown_connections 2022-11-27T16:04:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 --2- 172.21.15.105:0/4093078854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac240b3ae0 0x7fac240b3eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 --2- 172.21.15.105:0/4093078854 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac2415fdc0 0x7fac2406f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 -- 172.21.15.105:0/4093078854 >> 172.21.15.105:0/4093078854 conn(0x7fac240ceae0 msgr2=0x7fac240ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:01.068 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 -- 172.21.15.105:0/4093078854 shutdown_connections 2022-11-27T16:04:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.059+0000 7fac28ef3700 1 -- 172.21.15.105:0/4093078854 wait complete. 2022-11-27T16:04:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.061+0000 7fac28ef3700 1 Processor -- start 2022-11-27T16:04:01.069 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.061+0000 7fac28ef3700 1 -- start start 2022-11-27T16:04:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.061+0000 7fac28ef3700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac240b3ae0 0x7fac2415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.073 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.061+0000 7fac28ef3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac240d1540 0x7fac240cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.061+0000 7fac28ef3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac2415fdc0 0x7fac240c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.074 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac28ef3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac240cc3a0 con 0x7fac2415fdc0 2022-11-27T16:04:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac28ef3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2415d6f0 con 0x7fac240d1540 2022-11-27T16:04:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac28ef3700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2415d9d0 con 0x7fac240b3ae0 2022-11-27T16:04:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac2415fdc0 0x7fac240c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.075 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac2415fdc0 0x7fac240c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:51998/0 (socket says 172.21.15.105:51998) 2022-11-27T16:04:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 -- 172.21.15.105:0/771256083 learned_addr learned my addr 172.21.15.105:0/771256083 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac2259c700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac240b3ae0 0x7fac2415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.076 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac21d9b700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac240d1540 0x7fac240cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 -- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac240b3ae0 msgr2=0x7fac2415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac240b3ae0 0x7fac2415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 -- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac240d1540 msgr2=0x7fac240cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.077 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac240d1540 0x7fac240cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 -- 172.21.15.105:0/771256083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18009550 con 0x7fac2415fdc0 2022-11-27T16:04:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac2259c700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac240b3ae0 0x7fac2415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:04:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac21d9b700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac240d1540 0x7fac240cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:04:01.078 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac22d9d700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac2415fdc0 0x7fac240c8320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00cb50 tx=0x7fac1c00cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac12ffd700 1 -- 172.21.15.105:0/771256083 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c003c00 con 0x7fac2415fdc0 2022-11-27T16:04:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac12ffd700 1 -- 172.21.15.105:0/771256083 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c003d60 con 0x7fac2415fdc0 2022-11-27T16:04:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2415dcb0 con 0x7fac2415fdc0 2022-11-27T16:04:01.079 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac12ffd700 1 -- 172.21.15.105:0/771256083 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c00ab70 con 0x7fac2415fdc0 2022-11-27T16:04:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.062+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2415e270 con 0x7fac2415fdc0 2022-11-27T16:04:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.063+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac240cc4e0 con 0x7fac2415fdc0 2022-11-27T16:04:01.080 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.064+0000 7fac12ffd700 1 -- 172.21.15.105:0/771256083 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fac1c010050 con 0x7fac2415fdc0 2022-11-27T16:04:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.064+0000 7fac12ffd700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fac00066540 0x7fac000689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.064+0000 7fac12ffd700 1 -- 172.21.15.105:0/771256083 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fac1c0832f0 con 0x7fac2415fdc0 2022-11-27T16:04:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.064+0000 7fac2259c700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fac00066540 0x7fac000689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.081 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.064+0000 7fac2259c700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fac00066540 0x7fac000689f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac18014040 tx=0x7fac18008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.082 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.068+0000 7fac12ffd700 1 -- 172.21.15.105:0/771256083 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fac1c050da0 con 0x7fac2415fdc0 2022-11-27T16:04:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.306+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fac24067420 con 0x7fac2415fdc0 2022-11-27T16:04:01.309 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.307+0000 7fac12ffd700 1 -- 172.21.15.105:0/771256083 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7fac1c0543c0 con 0x7fac2415fdc0 2022-11-27T16:04:01.310 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:04:01.310 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/2690132438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":2690132438},{"type":"v1","addr":"172.21.15.105:6835","nonce":2690132438}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/3784259439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":3784259439},{"type":"v1","addr":"172.21.15.105:6837","nonce":3784259439}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3837691289","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6833","nonce":3837691289},{"type":"v1","addr":"172.21.15.170:6835","nonce":3837691289}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6834/2029196380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":2029196380},{"type":"v1","addr":"172.21.15.170:6834","nonce":2029196380}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-11-27T16:04:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fac00066540 msgr2=0x7fac000689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.311 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fac00066540 0x7fac000689f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fac18014040 tx=0x7fac18008030 comp rx=0 tx=0).stop 2022-11-27T16:04:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac2415fdc0 msgr2=0x7fac240c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac2415fdc0 0x7fac240c8320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00cb50 tx=0x7fac1c00cf10 comp rx=0 tx=0).stop 2022-11-27T16:04:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 shutdown_connections 2022-11-27T16:04:01.312 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fac00066540 0x7fac000689f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fac240b3ae0 0x7fac2415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fac2415fdc0 0x7fac240c8320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 --2- 172.21.15.105:0/771256083 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fac240d1540 0x7fac240cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.313 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 >> 172.21.15.105:0/771256083 conn(0x7fac240ceae0 msgr2=0x7fac240b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:01.314 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 shutdown_connections 2022-11-27T16:04:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.309+0000 7fac28ef3700 1 -- 172.21.15.105:0/771256083 wait complete. 2022-11-27T16:04:01.315 INFO:teuthology.orchestra.run.smithi105.stderr:dumped fsmap epoch 3 2022-11-27T16:04:01.324 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:04:01.450 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.445+0000 7f6895a99700 1 Processor -- start 2022-11-27T16:04:01.450 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.445+0000 7f6895a99700 1 -- start start 2022-11-27T16:04:01.451 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f6895a99700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900cd350 0x7f68900cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.451 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f6895a99700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f689015c100 0x7f6890069390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.451 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f6895a99700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68900cdc60 con 0x7f68900c44f0 2022-11-27T16:04:01.451 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f6895a99700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900c9e00 con 0x7f689015c100 2022-11-27T16:04:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f6895a99700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900ca0e0 con 0x7f68900cd350 2022-11-27T16:04:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688effd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f68900c44f0 0x7f68900caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43570/0 (socket says 172.21.15.105:43570) 2022-11-27T16:04:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688ffff700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f689015c100 0x7f6890069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688effd700 1 -- 172.21.15.105:0/2479632590 learned_addr learned my addr 172.21.15.105:0/2479632590 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:04:01.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688f7fe700 1 --2- 172.21.15.105:0/2479632590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900cd350 0x7f68900cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688ffff700 1 -- 172.21.15.105:0/2479632590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900cd350 msgr2=0x7f68900cd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688ffff700 1 --2- 172.21.15.105:0/2479632590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900cd350 0x7f68900cd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688ffff700 1 -- 172.21.15.105:0/2479632590 >> v1:172.21.15.105:6789/0 conn(0x7f68900c44f0 legacy=0x7f68900caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.446+0000 7f688ffff700 1 -- 172.21.15.105:0/2479632590 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900ca3c0 con 0x7f689015c100 2022-11-27T16:04:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f688ffff700 1 --2- 172.21.15.105:0/2479632590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f689015c100 0x7f6890069390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6888009d90 tx=0x7f688800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cd3da41de123a2 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f688e7fc700 1 -- 172.21.15.105:0/2479632590 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800e070 con 0x7f689015c100 2022-11-27T16:04:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f688e7fc700 1 -- 172.21.15.105:0/2479632590 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688800bb20 con 0x7f689015c100 2022-11-27T16:04:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f688e7fc700 1 -- 172.21.15.105:0/2479632590 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888012430 con 0x7f689015c100 2022-11-27T16:04:01.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 -- 172.21.15.105:0/2479632590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f689015c100 msgr2=0x7f6890069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.455 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 --2- 172.21.15.105:0/2479632590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f689015c100 0x7f6890069390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6888009d90 tx=0x7f688800b330 comp rx=0 tx=0).stop 2022-11-27T16:04:01.455 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 -- 172.21.15.105:0/2479632590 shutdown_connections 2022-11-27T16:04:01.456 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 --2- 172.21.15.105:0/2479632590 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900cd350 0x7f68900cd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.456 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 --2- 172.21.15.105:0/2479632590 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f689015c100 0x7f6890069390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.456 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 -- 172.21.15.105:0/2479632590 >> 172.21.15.105:0/2479632590 conn(0x7f68900cb7f0 msgr2=0x7f68900cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 -- 172.21.15.105:0/2479632590 shutdown_connections 2022-11-27T16:04:01.457 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.447+0000 7f6895a99700 1 -- 172.21.15.105:0/2479632590 wait complete. 2022-11-27T16:04:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.448+0000 7f6895a99700 1 Processor -- start 2022-11-27T16:04:01.461 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.449+0000 7f6895a99700 1 -- start start 2022-11-27T16:04:01.462 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900c44f0 0x7f6890155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.462 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900cd350 0x7f6890151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.463 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f689015c100 0x7f6890157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.463 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890156d10 con 0x7f68900cd350 2022-11-27T16:04:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6890151b40 con 0x7f689015c100 2022-11-27T16:04:01.464 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6890156470 con 0x7f68900c44f0 2022-11-27T16:04:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900cd350 0x7f6890151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688f7fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900c44f0 0x7f6890155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688f7fe700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900c44f0 0x7f6890155e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.105:37878/0 (socket says 172.21.15.105:37878) 2022-11-27T16:04:01.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688ffff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f689015c100 0x7f6890157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900cd350 0x7f6890151600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52012/0 (socket says 172.21.15.105:52012) 2022-11-27T16:04:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688f7fe700 1 -- 172.21.15.105:0/3395266323 learned_addr learned my addr 172.21.15.105:0/3395266323 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 -- 172.21.15.105:0/3395266323 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900c44f0 msgr2=0x7f6890155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900c44f0 0x7f6890155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 -- 172.21.15.105:0/3395266323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f689015c100 msgr2=0x7f6890157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f689015c100 0x7f6890157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.467 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 -- 172.21.15.105:0/3395266323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888009a70 con 0x7f68900cd350 2022-11-27T16:04:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f688effd700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900cd350 0x7f6890151600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f688400c390 tx=0x7f688400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6877fff700 1 -- 172.21.15.105:0/3395266323 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6884005b40 con 0x7f68900cd350 2022-11-27T16:04:01.468 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6877fff700 1 -- 172.21.15.105:0/3395266323 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6884005ca0 con 0x7f68900cd350 2022-11-27T16:04:01.469 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6890156750 con 0x7f68900cd350 2022-11-27T16:04:01.469 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6877fff700 1 -- 172.21.15.105:0/3395266323 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68840072f0 con 0x7f68900cd350 2022-11-27T16:04:01.469 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.450+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68901eceb0 con 0x7f68900cd350 2022-11-27T16:04:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.451+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68901ed0a0 con 0x7f68900cd350 2022-11-27T16:04:01.470 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.452+0000 7f6877fff700 1 -- 172.21.15.105:0/3395266323 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7f6884019030 con 0x7f68900cd350 2022-11-27T16:04:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.452+0000 7f6877fff700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f6870066540 0x7f68700689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.471 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.452+0000 7f688f7fe700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f6870066540 0x7f68700689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.452+0000 7f6877fff700 1 -- 172.21.15.105:0/3395266323 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6884082550 con 0x7f68900cd350 2022-11-27T16:04:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.452+0000 7f688f7fe700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f6870066540 0x7f68700689f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6880009c00 tx=0x7f6880009210 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.472 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.456+0000 7f6877fff700 1 -- 172.21.15.105:0/3395266323 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f6884050000 con 0x7f68900cd350 2022-11-27T16:04:01.639 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.637+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6890067420 con 0x7f68900cd350 2022-11-27T16:04:01.640 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.638+0000 7f6877fff700 1 -- 172.21.15.105:0/3395266323 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+13803 (secure 0 0 0) 0x7f6884017090 con 0x7f68900cd350 2022-11-27T16:04:01.641 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:04:01.641 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":16,"fsid":"558b439a-8d62-4391-a123-f6ff0625ded9","created":"2022-11-27T16:03:46.292863+0000","modified":"2022-11-27T16:04:00.906057+0000","last_up_change":"2022-11-27T16:03:57.895226+0000","last_in_change":"2022-11-27T16:03:49.700155+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2022-11-27T16:03:58.470316+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2022-11-27T16:03:59.588131+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a877ed0a-6953-4642-9d56-f1eae305b6a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":112238},{"type":"v1","addr":"172.21.15.105:6801","nonce":112238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":112238},{"type":"v1","addr":"172.21.15.105:6803","nonce":112238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":112238},{"type":"v1","addr":"172.21.15.105:6807","nonce":112238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":112238},{"type":"v1","addr":"172.21.15.105:6805","nonce":112238}]},"public_addr":"172.21.15.105:6801/112238","cluster_addr":"172.21.15.105:6803/112238","heartbeat_back_addr":"172.21.15.105:6807/112238","heartbeat_front_addr":"172.21.15.105:6805/112238","state":["exists","up"]},{"osd":1,"uuid":"dae5370d-59b3-458c-9821-f88f47769ab8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":112239},{"type":"v1","addr":"172.21.15.105:6809","nonce":112239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":112239},{"type":"v1","addr":"172.21.15.105:6811","nonce":112239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":112239},{"type":"v1","addr":"172.21.15.105:6815","nonce":112239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":112239},{"type":"v1","addr":"172.21.15.105:6813","nonce":112239}]},"public_addr":"172.21.15.105:6809/112239","cluster_addr":"172.21.15.105:6811/112239","heartbeat_back_addr":"172.21.15.105:6815/112239","heartbeat_front_addr":"172.21.15.105:6813/112239","state":["exists","up"]},{"osd":2,"uuid":"e34db7ee-5ac2-4682-8b3d-77781aff8feb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":112240},{"type":"v1","addr":"172.21.15.105:6817","nonce":112240}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":112240},{"type":"v1","addr":"172.21.15.105:6819","nonce":112240}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":112240},{"type":"v1","addr":"172.21.15.105:6823","nonce":112240}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":112240},{"type":"v1","addr":"172.21.15.105:6821","nonce":112240}]},"public_addr":"172.21.15.105:6817/112240","cluster_addr":"172.21.15.105:6819/112240","heartbeat_back_addr":"172.21.15.105:6823/112240","heartbeat_front_addr":"172.21.15.105:6821/112240","state":["exists","up"]},{"osd":3,"uuid":"98f2fda2-a32e-4e5c-ad39-c3d67013ccd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":112241},{"type":"v1","addr":"172.21.15.105:6825","nonce":112241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":112241},{"type":"v1","addr":"172.21.15.105:6827","nonce":112241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":112241},{"type":"v1","addr":"172.21.15.105:6831","nonce":112241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":112241},{"type":"v1","addr":"172.21.15.105:6829","nonce":112241}]},"public_addr":"172.21.15.105:6825/112241","cluster_addr":"172.21.15.105:6827/112241","heartbeat_back_addr":"172.21.15.105:6831/112241","heartbeat_front_addr":"172.21.15.105:6829/112241","state":["exists","up"]},{"osd":4,"uuid":"ec8582a4-93d8-4e63-8b94-5386bf561c11","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":112474},{"type":"v1","addr":"172.21.15.170:6801","nonce":112474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":112474},{"type":"v1","addr":"172.21.15.170:6803","nonce":112474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":112474},{"type":"v1","addr":"172.21.15.170:6807","nonce":112474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":112474},{"type":"v1","addr":"172.21.15.170:6805","nonce":112474}]},"public_addr":"172.21.15.170:6801/112474","cluster_addr":"172.21.15.170:6803/112474","heartbeat_back_addr":"172.21.15.170:6807/112474","heartbeat_front_addr":"172.21.15.170:6805/112474","state":["exists","up"]},{"osd":5,"uuid":"55b1b3a8-5dac-4f1e-b0d6-1871422ae097","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":112486},{"type":"v1","addr":"172.21.15.170:6818","nonce":112486}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6819","nonce":112486},{"type":"v1","addr":"172.21.15.170:6821","nonce":112486}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":112486},{"type":"v1","addr":"172.21.15.170:6827","nonce":112486}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6823","nonce":112486},{"type":"v1","addr":"172.21.15.170:6824","nonce":112486}]},"public_addr":"172.21.15.170:6818/112486","cluster_addr":"172.21.15.170:6821/112486","heartbeat_back_addr":"172.21.15.170:6827/112486","heartbeat_front_addr":"172.21.15.170:6824/112486","state":["exists","up"]},{"osd":6,"uuid":"64ff9585-481c-4ab8-a276-dad128c5dba9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":112475},{"type":"v1","addr":"172.21.15.170:6809","nonce":112475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":112475},{"type":"v1","addr":"172.21.15.170:6811","nonce":112475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":112475},{"type":"v1","addr":"172.21.15.170:6815","nonce":112475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":112475},{"type":"v1","addr":"172.21.15.170:6813","nonce":112475}]},"public_addr":"172.21.15.170:6809/112475","cluster_addr":"172.21.15.170:6811/112475","heartbeat_back_addr":"172.21.15.170:6815/112475","heartbeat_front_addr":"172.21.15.170:6813/112475","state":["exists","up"]},{"osd":7,"uuid":"9a2592f7-a19d-49df-beb3-004cd10cf16b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6817","nonce":112495},{"type":"v1","addr":"172.21.15.170:6820","nonce":112495}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":112495},{"type":"v1","addr":"172.21.15.170:6825","nonce":112495}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":112495},{"type":"v1","addr":"172.21.15.170:6831","nonce":112495}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":112495},{"type":"v1","addr":"172.21.15.170:6829","nonce":112495}]},"public_addr":"172.21.15.170:6820/112495","cluster_addr":"172.21.15.170:6825/112495","heartbeat_back_addr":"172.21.15.170:6831/112495","heartbeat_front_addr":"172.21.15.170:6829/112495","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.603339+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.577029+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.764972+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.940421+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.831837+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.894911+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.805503+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:03:54.855604+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:04:01.642 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f6870066540 msgr2=0x7f68700689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.643 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f6870066540 0x7f68700689f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6880009c00 tx=0x7f6880009210 comp rx=0 tx=0).stop 2022-11-27T16:04:01.643 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900cd350 msgr2=0x7f6890151600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.643 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900cd350 0x7f6890151600 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f688400c390 tx=0x7f688400c6a0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.644 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 shutdown_connections 2022-11-27T16:04:01.644 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f6870066540 0x7f68700689f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.644 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f68900c44f0 0x7f6890155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.644 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f689015c100 0x7f6890157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 --2- 172.21.15.105:0/3395266323 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f68900cd350 0x7f6890151600 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 >> 172.21.15.105:0/3395266323 conn(0x7f68900cb7f0 msgr2=0x7f6890069420 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 shutdown_connections 2022-11-27T16:04:01.645 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.640+0000 7f6895a99700 1 -- 172.21.15.105:0/3395266323 wait complete. 2022-11-27T16:04:01.654 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:03:58.470316+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:03:59.588131+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2022-11-27T16:04:01.655 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:04:01.783 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.781+0000 7feb00d52700 1 Processor -- start 2022-11-27T16:04:01.784 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.781+0000 7feb00d52700 1 -- start start 2022-11-27T16:04:01.789 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feb00d52700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc0b3ae0 0x7feafc0b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feb00d52700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7feafc0d1540 0x7feafc0700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feb00d52700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feafc0cc950 con 0x7feafc15fdc0 2022-11-27T16:04:01.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feb00d52700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feafc0cc6b0 con 0x7feafc0d1540 2022-11-27T16:04:01.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feb00d52700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0ccde0 con 0x7feafc0b3ae0 2022-11-27T16:04:01.791 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feaf9d9b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feafc15fdc0 0x7feafc06f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43576/0 (socket says 172.21.15.105:43576) 2022-11-27T16:04:01.791 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feaf9d9b700 1 -- 172.21.15.105:0/644116551 learned_addr learned my addr 172.21.15.105:0/644116551 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:04:01.791 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feafa59c700 1 --2- 172.21.15.105:0/644116551 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc0b3ae0 0x7feafc0b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.791 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feafad9d700 1 --2- 172.21.15.105:0/644116551 >> v2:172.21.15.170:3300/0 conn(0x7feafc0d1540 0x7feafc0700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.791 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feaf959a700 1 -- 172.21.15.105:0/644116551 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667014818 0 0) 0x7feafc0cc950 con 0x7feafc15fdc0 2022-11-27T16:04:01.792 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feaf959a700 1 -- 172.21.15.105:0/644116551 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae00035b0 con 0x7feafc15fdc0 2022-11-27T16:04:01.792 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feafa59c700 1 -- 172.21.15.105:0/644116551 >> v2:172.21.15.170:3300/0 conn(0x7feafc0d1540 msgr2=0x7feafc0700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:01.792 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feafa59c700 1 --2- 172.21.15.105:0/644116551 >> v2:172.21.15.170:3300/0 conn(0x7feafc0d1540 0x7feafc0700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.792 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feafa59c700 1 -- 172.21.15.105:0/644116551 >> v1:172.21.15.105:6789/0 conn(0x7feafc15fdc0 legacy=0x7feafc06f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.793 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feaf959a700 1 -- 172.21.15.105:0/644116551 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3626119947 0 0) 0x7feae00035b0 con 0x7feafc15fdc0 2022-11-27T16:04:01.793 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feafa59c700 1 -- 172.21.15.105:0/644116551 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc1602c0 con 0x7feafc0b3ae0 2022-11-27T16:04:01.793 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.782+0000 7feafa59c700 1 --2- 172.21.15.105:0/644116551 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc0b3ae0 0x7feafc0b3eb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009840 tx=0x7feaf0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ef05917af12d364 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.793 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feaf959a700 1 -- 172.21.15.105:0/644116551 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec001670 con 0x7feafc0b3ae0 2022-11-27T16:04:01.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feaf959a700 1 -- 172.21.15.105:0/644116551 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf0004030 con 0x7feafc0b3ae0 2022-11-27T16:04:01.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feaf959a700 1 -- 172.21.15.105:0/644116551 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf00043c0 con 0x7feafc0b3ae0 2022-11-27T16:04:01.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 -- 172.21.15.105:0/644116551 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc0b3ae0 msgr2=0x7feafc0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 --2- 172.21.15.105:0/644116551 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc0b3ae0 0x7feafc0b3eb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7feaf0009840 tx=0x7feaf0009b50 comp rx=0 tx=0).stop 2022-11-27T16:04:01.795 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 -- 172.21.15.105:0/644116551 shutdown_connections 2022-11-27T16:04:01.795 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 --2- 172.21.15.105:0/644116551 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc0b3ae0 0x7feafc0b3eb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.795 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 --2- 172.21.15.105:0/644116551 >> v2:172.21.15.170:3300/0 conn(0x7feafc0d1540 0x7feafc0700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.796 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 -- 172.21.15.105:0/644116551 >> 172.21.15.105:0/644116551 conn(0x7feafc0ceae0 msgr2=0x7feafc0ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:01.796 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 -- 172.21.15.105:0/644116551 shutdown_connections 2022-11-27T16:04:01.796 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.783+0000 7feb00d52700 1 -- 172.21.15.105:0/644116551 wait complete. 2022-11-27T16:04:01.796 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 Processor -- start 2022-11-27T16:04:01.797 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 -- start start 2022-11-27T16:04:01.797 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feafc0b3ae0 0x7feafc15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.797 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feafc0d1540 0x7feafc0c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.797 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc15fdc0 0x7feafc0c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.797 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0c7e10 con 0x7feafc0d1540 2022-11-27T16:04:01.798 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0c9b90 con 0x7feafc0b3ae0 2022-11-27T16:04:01.798 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.785+0000 7feb00d52700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0c8030 con 0x7feafc15fdc0 2022-11-27T16:04:01.798 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feafc0d1540 0x7feafc0c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.798 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feafc0d1540 0x7feafc0c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52022/0 (socket says 172.21.15.105:52022) 2022-11-27T16:04:01.799 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 -- 172.21.15.105:0/583003031 learned_addr learned my addr 172.21.15.105:0/583003031 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:01.799 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feafa59c700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feafc0b3ae0 0x7feafc15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.799 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feafad9d700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc15fdc0 0x7feafc0c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.799 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 -- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc15fdc0 msgr2=0x7feafc0c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.799 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc15fdc0 0x7feafc0c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.800 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 -- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feafc0b3ae0 msgr2=0x7feafc15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.800 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feafc0b3ae0 0x7feafc15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.800 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 -- 172.21.15.105:0/583003031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00094a0 con 0x7feafc0d1540 2022-11-27T16:04:01.800 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaf9d9b700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feafc0d1540 0x7feafc0c7390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c750 tx=0x7feaec00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.801 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaeaffd700 1 -- 172.21.15.105:0/583003031 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec004c80 con 0x7feafc0d1540 2022-11-27T16:04:01.801 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaeaffd700 1 -- 172.21.15.105:0/583003031 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec004de0 con 0x7feafc0d1540 2022-11-27T16:04:01.801 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc0c9260 con 0x7feafc0d1540 2022-11-27T16:04:01.801 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feaeaffd700 1 -- 172.21.15.105:0/583003031 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0072d0 con 0x7feafc0d1540 2022-11-27T16:04:01.801 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.786+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc0c9820 con 0x7feafc0d1540 2022-11-27T16:04:01.802 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.787+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feafc067420 con 0x7feafc0d1540 2022-11-27T16:04:01.802 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.787+0000 7feaeaffd700 1 -- 172.21.15.105:0/583003031 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7feaec01a020 con 0x7feafc0d1540 2022-11-27T16:04:01.803 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.788+0000 7feaeaffd700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fead8066540 0x7fead80689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:01.803 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.788+0000 7feaeaffd700 1 -- 172.21.15.105:0/583003031 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5206+0+0 (secure 0 0 0) 0x7feaec082fe0 con 0x7feafc0d1540 2022-11-27T16:04:01.804 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.788+0000 7feafa59c700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fead8066540 0x7fead80689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:01.804 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.788+0000 7feafa59c700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fead8066540 0x7fead80689f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feaf00090b0 tx=0x7feaf0008030 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:01.804 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.792+0000 7feaeaffd700 1 -- 172.21.15.105:0/583003031 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7feaec050a10 con 0x7feafc0d1540 2022-11-27T16:04:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.976+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feafc0c8740 con 0x7feafc0d1540 2022-11-27T16:04:01.979 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.976+0000 7feaeaffd700 1 -- 172.21.15.105:0/583003031 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7feaec054030 con 0x7feafc0d1540 2022-11-27T16:04:01.980 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 8 2022-11-27T16:04:01.982 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fead8066540 msgr2=0x7fead80689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.983 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fead8066540 0x7fead80689f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feaf00090b0 tx=0x7feaf0008030 comp rx=0 tx=0).stop 2022-11-27T16:04:01.983 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feafc0d1540 msgr2=0x7feafc0c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feafc0d1540 0x7feafc0c7390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7feaec00c750 tx=0x7feaec00ca60 comp rx=0 tx=0).stop 2022-11-27T16:04:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 shutdown_connections 2022-11-27T16:04:01.984 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fead8066540 0x7fead80689f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7feafc15fdc0 0x7feafc0c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.985 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feafc0b3ae0 0x7feafc15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.986 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 --2- 172.21.15.105:0/583003031 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7feafc0d1540 0x7feafc0c7390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:01.986 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 >> 172.21.15.105:0/583003031 conn(0x7feafc0ceae0 msgr2=0x7feafc0d0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 shutdown_connections 2022-11-27T16:04:01.987 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:01.978+0000 7feb00d52700 1 -- 172.21.15.105:0/583003031 wait complete. 2022-11-27T16:04:01.993 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:04:02.118 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.113+0000 7fe05a358700 1 Processor -- start 2022-11-27T16:04:02.119 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.113+0000 7fe05a358700 1 -- start start 2022-11-27T16:04:02.119 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.114+0000 7fe05a358700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0540ccc80 0x7fe0540cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.119 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.114+0000 7fe05a358700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe05406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.120 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.114+0000 7fe05a358700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05406ef00 con 0x7fe0540cc0e0 2022-11-27T16:04:02.120 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.114+0000 7fe05a358700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0540b2800 con 0x7fe0540ccc80 2022-11-27T16:04:02.120 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.114+0000 7fe05a358700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540b1cf0 con 0x7fe0540b18d0 2022-11-27T16:04:02.120 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.114+0000 7fe0588f5700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe05406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.114+0000 7fe053fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0540ccc80 0x7fe0540cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe053fff700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7fe0540ccc80 0x7fe0540cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.105:43962/0 (socket says 172.21.15.105:43962) 2022-11-27T16:04:02.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe0588f5700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe05406a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3301/0 says I am v2:172.21.15.105:37936/0 (socket says 172.21.15.105:37936) 2022-11-27T16:04:02.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe053fff700 1 -- 172.21.15.105:0/1062337182 learned_addr learned my addr 172.21.15.105:0/1062337182 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:02.121 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe0588f5700 1 -- 172.21.15.105:0/1062337182 >> v2:172.21.15.170:3300/0 conn(0x7fe0540ccc80 msgr2=0x7fe0540cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe0588f5700 1 --2- 172.21.15.105:0/1062337182 >> v2:172.21.15.170:3300/0 conn(0x7fe0540ccc80 0x7fe0540cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe0588f5700 1 -- 172.21.15.105:0/1062337182 >> v1:172.21.15.170:6789/0 conn(0x7fe0540cc0e0 legacy=0x7fe05415fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe0588f5700 1 -- 172.21.15.105:0/1062337182 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0540b1f60 con 0x7fe0540b18d0 2022-11-27T16:04:02.122 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe053fff700 1 --2- 172.21.15.105:0/1062337182 >> v2:172.21.15.170:3300/0 conn(0x7fe0540ccc80 0x7fe0540cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:04:02.123 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe0588f5700 1 --2- 172.21.15.105:0/1062337182 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe05406a5b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009d90 tx=0x7fe04c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fe37d59c0e7fe70 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.123 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe052ffd700 1 -- 172.21.15.105:0/1062337182 <== mon.2 v2:172.21.15.170:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c00e070 con 0x7fe0540b18d0 2022-11-27T16:04:02.123 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe052ffd700 1 -- 172.21.15.105:0/1062337182 <== mon.2 v2:172.21.15.170:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04c00bb20 con 0x7fe0540b18d0 2022-11-27T16:04:02.123 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.115+0000 7fe052ffd700 1 -- 172.21.15.105:0/1062337182 <== mon.2 v2:172.21.15.170:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04c012430 con 0x7fe0540b18d0 2022-11-27T16:04:02.124 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 -- 172.21.15.105:0/1062337182 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 msgr2=0x7fe05406a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.124 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 --2- 172.21.15.105:0/1062337182 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe05406a5b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009d90 tx=0x7fe04c00b330 comp rx=0 tx=0).stop 2022-11-27T16:04:02.124 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 -- 172.21.15.105:0/1062337182 shutdown_connections 2022-11-27T16:04:02.125 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 --2- 172.21.15.105:0/1062337182 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe05406a5b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.125 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 --2- 172.21.15.105:0/1062337182 >> v2:172.21.15.170:3300/0 conn(0x7fe0540ccc80 0x7fe0540cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.126 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 -- 172.21.15.105:0/1062337182 >> 172.21.15.105:0/1062337182 conn(0x7fe05406cfe0 msgr2=0x7fe05406d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:02.126 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 -- 172.21.15.105:0/1062337182 shutdown_connections 2022-11-27T16:04:02.126 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.116+0000 7fe05a358700 1 -- 172.21.15.105:0/1062337182 wait complete. 2022-11-27T16:04:02.129 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.117+0000 7fe05a358700 1 Processor -- start 2022-11-27T16:04:02.130 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.117+0000 7fe05a358700 1 -- start start 2022-11-27T16:04:02.130 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe05a358700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe054150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.130 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe05a358700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0540cc0e0 0x7fe0540c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.130 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe05a358700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0540ccc80 0x7fe0541f0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.131 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe05a358700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540c44a0 con 0x7fe0540cc0e0 2022-11-27T16:04:02.131 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe05a358700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415a9f0 con 0x7fe0540ccc80 2022-11-27T16:04:02.131 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe05a358700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540c3b00 con 0x7fe0540b18d0 2022-11-27T16:04:02.131 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0540cc0e0 0x7fe0540c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0540cc0e0 0x7fe0540c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52034/0 (socket says 172.21.15.105:52034) 2022-11-27T16:04:02.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 -- 172.21.15.105:0/849622701 learned_addr learned my addr 172.21.15.105:0/849622701 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:02.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe053fff700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe054150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0588f5700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0540ccc80 0x7fe0541f0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.132 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 -- 172.21.15.105:0/849622701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 msgr2=0x7fe054150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe054150480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 -- 172.21.15.105:0/849622701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0540ccc80 msgr2=0x7fe0541f0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.133 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0540ccc80 0x7fe0541f0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 -- 172.21.15.105:0/849622701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c009550 con 0x7fe0540cc0e0 2022-11-27T16:04:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe0537fe700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0540cc0e0 0x7fe0540c5c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe04400c450 tx=0x7fe04400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe050ff9700 1 -- 172.21.15.105:0/849622701 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0440052d0 con 0x7fe0540cc0e0 2022-11-27T16:04:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe050ff9700 1 -- 172.21.15.105:0/849622701 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe044005430 con 0x7fe0540cc0e0 2022-11-27T16:04:02.134 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe050ff9700 1 -- 172.21.15.105:0/849622701 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0440038c0 con 0x7fe0540cc0e0 2022-11-27T16:04:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.118+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c009a70 con 0x7fe0540cc0e0 2022-11-27T16:04:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.119+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0540c40c0 con 0x7fe0540cc0e0 2022-11-27T16:04:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.120+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0540c45e0 con 0x7fe0540cc0e0 2022-11-27T16:04:02.135 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.120+0000 7fe050ff9700 1 -- 172.21.15.105:0/849622701 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7fe044019030 con 0x7fe0540cc0e0 2022-11-27T16:04:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.120+0000 7fe050ff9700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fe038066540 0x7fe0380689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.120+0000 7fe050ff9700 1 -- 172.21.15.105:0/849622701 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe044082570 con 0x7fe0540cc0e0 2022-11-27T16:04:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.120+0000 7fe053fff700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fe038066540 0x7fe0380689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.121+0000 7fe053fff700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fe038066540 0x7fe0380689f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe03c006d30 tx=0x7fe03c0092c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.136 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.124+0000 7fe050ff9700 1 -- 172.21.15.105:0/849622701 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fe0540c45e0 con 0x7fe0540cc0e0 2022-11-27T16:04:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.299+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0540c45e0 con 0x7fe0540cc0e0 2022-11-27T16:04:02.302 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.300+0000 7fe050ff9700 1 -- 172.21.15.105:0/849622701 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe044017020 con 0x7fe0540cc0e0 2022-11-27T16:04:02.302 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2022-11-27T16:04:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fe038066540 msgr2=0x7fe0380689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.304 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fe038066540 0x7fe0380689f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe03c006d30 tx=0x7fe03c0092c0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.305 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0540cc0e0 msgr2=0x7fe0540c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.305 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0540cc0e0 0x7fe0540c5c70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe04400c450 tx=0x7fe04400c760 comp rx=0 tx=0).stop 2022-11-27T16:04:02.305 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 shutdown_connections 2022-11-27T16:04:02.305 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7fe038066540 0x7fe0380689f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.306 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7fe0540b18d0 0x7fe054150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.306 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0540ccc80 0x7fe0541f0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.306 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 --2- 172.21.15.105:0/849622701 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe0540cc0e0 0x7fe0540c5c70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.306 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 >> 172.21.15.105:0/849622701 conn(0x7fe05406cfe0 msgr2=0x7fe05415b290 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:02.306 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 shutdown_connections 2022-11-27T16:04:02.307 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.302+0000 7fe05a358700 1 -- 172.21.15.105:0/849622701 wait complete. 2022-11-27T16:04:02.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T16:04:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.437+0000 7f3aa9fab700 1 Processor -- start 2022-11-27T16:04:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.437+0000 7f3aa9fab700 1 -- start start 2022-11-27T16:04:02.451 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa9fab700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f3aa40ad540 0x7f3aa40ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa9fab700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 0x7f3aa40abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa9fab700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa4158c30 con 0x7f3aa4158860 2022-11-27T16:04:02.452 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa9fab700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa415d080 con 0x7f3aa40ad540 2022-11-27T16:04:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa9fab700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40ab090 con 0x7f3aa40c9220 2022-11-27T16:04:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa37fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3aa4158860 0x7f3aa40c8b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43582/0 (socket says 172.21.15.105:43582) 2022-11-27T16:04:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa37fe700 1 -- 172.21.15.105:0/2874816086 learned_addr learned my addr 172.21.15.105:0/2874816086 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:04:02.453 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093788879 0 0) 0x7f3a94001290 con 0x7f3aa4158860 2022-11-27T16:04:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/2874816086 >> v2:172.21.15.170:3300/0 conn(0x7f3aa40ad540 0x7f3aa40ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a900035b0 con 0x7f3aa4158860 2022-11-27T16:04:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa3fff700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 0x7f3aa40abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 843104868 0 0) 0x7f3a900035b0 con 0x7f3aa4158860 2022-11-27T16:04:02.454 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 msgr2=0x7f3aa40abe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:02.455 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 0x7f3aa40abe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.455 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 >> v2:172.21.15.170:3300/0 conn(0x7f3aa40ad540 msgr2=0x7f3aa40ad9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:02.455 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 --2- 172.21.15.105:0/2874816086 >> v2:172.21.15.170:3300/0 conn(0x7f3aa40ad540 0x7f3aa40ad9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.455 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa40ab370 con 0x7f3aa4158860 2022-11-27T16:04:02.456 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3018764546 0 0) 0x7f3a94001980 con 0x7f3aa4158860 2022-11-27T16:04:02.456 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 >> v1:172.21.15.105:6789/0 conn(0x7f3aa4158860 legacy=0x7f3aa40c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.456 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 0x7f3a90003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a90003f80 0x7f3a90004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90004d90 0x7f3a90005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.457 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa415d080 con 0x7f3a90003f80 2022-11-27T16:04:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40ab090 con 0x7f3a90004d90 2022-11-27T16:04:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.438+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000ba10 con 0x7f3aa40c9220 2022-11-27T16:04:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a90003f80 0x7f3a90004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa37fe700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 0x7f3a90003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.458 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa3fff700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90004d90 0x7f3a90005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa2ffd700 1 -- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 msgr2=0x7f3a90003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40c9220 0x7f3a90003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.459 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa2ffd700 1 -- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90004d90 msgr2=0x7f3a90005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90004d90 0x7f3a90005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa2ffd700 1 -- 172.21.15.105:0/2874816086 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a900035b0 con 0x7f3a90003f80 2022-11-27T16:04:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a90003f80 0x7f3a90004850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40adef0 tx=0x7f3a90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6439d3660fe5fc89 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.460 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800e6c0 con 0x7f3a90003f80 2022-11-27T16:04:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9800e820 con 0x7f3a90003f80 2022-11-27T16:04:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.439+0000 7f3aa27fc700 1 -- 172.21.15.105:0/2874816086 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800eb90 con 0x7f3a90003f80 2022-11-27T16:04:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.440+0000 7f3aa9fab700 1 -- 172.21.15.105:0/2874816086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a90003f80 msgr2=0x7f3a90004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.461 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.440+0000 7f3aa9fab700 1 --2- 172.21.15.105:0/2874816086 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a90003f80 0x7f3a90004850 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3aa40adef0 tx=0x7f3a90005880 comp rx=0 tx=0).stop 2022-11-27T16:04:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.440+0000 7f3aa3fff700 1 -- 172.21.15.105:0/2874816086 reap_dead start 2022-11-27T16:04:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.440+0000 7f3aa9fab700 1 -- 172.21.15.105:0/2874816086 shutdown_connections 2022-11-27T16:04:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.440+0000 7f3aa9fab700 1 -- 172.21.15.105:0/2874816086 >> 172.21.15.105:0/2874816086 conn(0x7f3aa4070a30 msgr2=0x7f3aa4070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.440+0000 7f3aa9fab700 1 -- 172.21.15.105:0/2874816086 shutdown_connections 2022-11-27T16:04:02.462 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.440+0000 7f3aa9fab700 1 -- 172.21.15.105:0/2874816086 wait complete. 2022-11-27T16:04:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.441+0000 7f3aa9fab700 1 Processor -- start 2022-11-27T16:04:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.441+0000 7f3aa9fab700 1 -- start start 2022-11-27T16:04:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90003f80 0x7f3aa40cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4157c00 0x7f3aa41f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40abd50 0x7f3aa40ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40ac730 con 0x7f3aa4157c00 2022-11-27T16:04:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa40cd900 con 0x7f3a90003f80 2022-11-27T16:04:02.464 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa41f25e0 con 0x7f3aa40abd50 2022-11-27T16:04:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4157c00 0x7f3aa41f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4157c00 0x7f3aa41f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52046/0 (socket says 172.21.15.105:52046) 2022-11-27T16:04:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 -- 172.21.15.105:0/3536214904 learned_addr learned my addr 172.21.15.105:0/3536214904 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:02.465 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa3fff700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40abd50 0x7f3aa40ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa37fe700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90003f80 0x7f3aa40cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 -- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40abd50 msgr2=0x7f3aa40ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40abd50 0x7f3aa40ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.466 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 -- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90003f80 msgr2=0x7f3aa40cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90003f80 0x7f3aa40cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 -- 172.21.15.105:0/3536214904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4158070 con 0x7f3aa4157c00 2022-11-27T16:04:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa2ffd700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4157c00 0x7f3aa41f20a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a98000c00 tx=0x7f3a980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.467 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9800e6c0 con 0x7f3aa4157c00 2022-11-27T16:04:02.468 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98003ad0 con 0x7f3aa4157c00 2022-11-27T16:04:02.468 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98006e60 con 0x7f3aa4157c00 2022-11-27T16:04:02.468 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa40cd340 con 0x7f3aa4157c00 2022-11-27T16:04:02.468 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.442+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa40af2d0 con 0x7f3aa4157c00 2022-11-27T16:04:02.469 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.443+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aa4067420 con 0x7f3aa4157c00 2022-11-27T16:04:02.469 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.444+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7f3a98019030 con 0x7f3aa4157c00 2022-11-27T16:04:02.469 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.444+0000 7f3a8bfff700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f3a84066530 0x7f3a840689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.469 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.444+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f3a98084b80 con 0x7f3aa4157c00 2022-11-27T16:04:02.470 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.444+0000 7f3aa37fe700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f3a84066530 0x7f3a840689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.470 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.444+0000 7f3aa37fe700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f3a84066530 0x7f3a840689e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a94000c00 tx=0x7f3a94007b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.470 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.448+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f3a98052630 con 0x7f3aa4157c00 2022-11-27T16:04:02.471 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.457+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f3a98052a00 con 0x7f3aa4157c00 2022-11-27T16:04:02.634 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.632+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3aa40ab650 con 0x7f3aa4157c00 2022-11-27T16:04:02.635 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.633+0000 7f3a8bfff700 1 -- 172.21.15.105:0/3536214904 <== mon.0 v2:172.21.15.105:3300/0 8 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3a980309c0 con 0x7f3aa4157c00 2022-11-27T16:04:02.635 INFO:teuthology.orchestra.run.smithi105.stdout: 2022-11-27T16:04:02.636 INFO:teuthology.orchestra.run.smithi105.stdout:[ 2022-11-27T16:04:02.636 INFO:teuthology.orchestra.run.smithi105.stdout: { 2022-11-27T16:04:02.636 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 1, 2022-11-27T16:04:02.636 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": "rbd" 2022-11-27T16:04:02.637 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2022-11-27T16:04:02.637 INFO:teuthology.orchestra.run.smithi105.stdout: { 2022-11-27T16:04:02.637 INFO:teuthology.orchestra.run.smithi105.stdout: "poolnum": 2, 2022-11-27T16:04:02.637 INFO:teuthology.orchestra.run.smithi105.stdout: "poolname": ".mgr" 2022-11-27T16:04:02.638 INFO:teuthology.orchestra.run.smithi105.stdout: } 2022-11-27T16:04:02.638 INFO:teuthology.orchestra.run.smithi105.stdout:] 2022-11-27T16:04:02.639 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f3a84066530 msgr2=0x7f3a840689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f3a84066530 0x7f3a840689e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a94000c00 tx=0x7f3a94007b20 comp rx=0 tx=0).stop 2022-11-27T16:04:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4157c00 msgr2=0x7f3aa41f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4157c00 0x7f3aa41f20a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3a98000c00 tx=0x7f3a980047c0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.640 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 shutdown_connections 2022-11-27T16:04:02.641 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f3a84066530 0x7f3a840689e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.641 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f3aa40abd50 0x7f3aa40ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.641 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3a90003f80 0x7f3aa40cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.641 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 --2- 172.21.15.105:0/3536214904 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3aa4157c00 0x7f3aa41f20a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.642 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.634+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 >> 172.21.15.105:0/3536214904 conn(0x7f3aa4070a30 msgr2=0x7f3aa40b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:02.642 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.635+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 shutdown_connections 2022-11-27T16:04:02.642 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.635+0000 7f3aa9fab700 1 -- 172.21.15.105:0/3536214904 wait complete. 2022-11-27T16:04:02.648 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T16:04:02.649 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T16:04:02.773 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.770+0000 7f7f938c5700 1 Processor -- start 2022-11-27T16:04:02.773 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.771+0000 7f7f938c5700 1 -- start start 2022-11-27T16:04:02.776 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.771+0000 7f7f938c5700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7f8c0ae6b0 0x7f7f8c0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.777 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.771+0000 7f7f938c5700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c0cd620 con 0x7f7f8c15bd60 2022-11-27T16:04:02.777 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.771+0000 7f7f938c5700 1 -- --> v1:172.21.15.170:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f8c0cb8a0 con 0x7f7f8c0cf120 2022-11-27T16:04:02.777 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.771+0000 7f7f938c5700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f8c0ccd30 con 0x7f7f8c0ae6b0 2022-11-27T16:04:02.777 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f90e60700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f7f8c0ae6b0 0x7f7f8c0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.778 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f91661700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f7f8c15bd60 0x7f7f8c0cdab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43594/0 (socket says 172.21.15.105:43594) 2022-11-27T16:04:02.778 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f91661700 1 -- 172.21.15.105:0/2778377523 learned_addr learned my addr 172.21.15.105:0/2778377523 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:04:02.778 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f90e60700 1 -- 172.21.15.105:0/2778377523 >> v1:172.21.15.170:6789/0 conn(0x7f7f8c0cf120 legacy=0x7f7f8c0aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.779 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f90e60700 1 -- 172.21.15.105:0/2778377523 >> v1:172.21.15.105:6789/0 conn(0x7f7f8c15bd60 legacy=0x7f7f8c0cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.779 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f90e60700 1 -- 172.21.15.105:0/2778377523 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c0ae290 con 0x7f7f8c0ae6b0 2022-11-27T16:04:02.779 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f7bfff700 1 -- 172.21.15.105:0/2778377523 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 891735872 0 0) 0x7f7f8c0cd620 con 0x7f7f8c15bd60 2022-11-27T16:04:02.780 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f90e60700 1 --2- 172.21.15.105:0/2778377523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c0ae6b0 0x7f7f8c0c8b30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009840 tx=0x7f7f88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efa9d8ff314061e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.780 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f7bfff700 1 -- 172.21.15.105:0/2778377523 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88011830 con 0x7f7f8c0ae6b0 2022-11-27T16:04:02.780 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f7bfff700 1 -- 172.21.15.105:0/2778377523 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88011990 con 0x7f7f8c0ae6b0 2022-11-27T16:04:02.780 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.772+0000 7f7f7bfff700 1 -- 172.21.15.105:0/2778377523 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88011d20 con 0x7f7f8c0ae6b0 2022-11-27T16:04:02.781 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.773+0000 7f7f938c5700 1 -- 172.21.15.105:0/2778377523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c0ae6b0 msgr2=0x7f7f8c0c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.781 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.773+0000 7f7f938c5700 1 --2- 172.21.15.105:0/2778377523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c0ae6b0 0x7f7f8c0c8b30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009840 tx=0x7f7f88009b50 comp rx=0 tx=0).stop 2022-11-27T16:04:02.781 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.773+0000 7f7f938c5700 1 -- 172.21.15.105:0/2778377523 shutdown_connections 2022-11-27T16:04:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.773+0000 7f7f938c5700 1 --2- 172.21.15.105:0/2778377523 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c0ae6b0 0x7f7f8c0c8b30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.773+0000 7f7f938c5700 1 -- 172.21.15.105:0/2778377523 >> 172.21.15.105:0/2778377523 conn(0x7f7f8c069cb0 msgr2=0x7f7f8c0ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.773+0000 7f7f938c5700 1 -- 172.21.15.105:0/2778377523 shutdown_connections 2022-11-27T16:04:02.782 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.773+0000 7f7f938c5700 1 -- 172.21.15.105:0/2778377523 wait complete. 2022-11-27T16:04:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.774+0000 7f7f938c5700 1 Processor -- start 2022-11-27T16:04:02.783 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.774+0000 7f7f938c5700 1 -- start start 2022-11-27T16:04:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f938c5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f8c0ae6b0 0x7f7f8c1f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.786 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f938c5700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f8c0cf120 0x7f7f8c15c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f938c5700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c15bd60 0x7f7f8c15e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f938c5700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15c670 con 0x7f7f8c15bd60 2022-11-27T16:04:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f938c5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c15ec80 con 0x7f7f8c0ae6b0 2022-11-27T16:04:02.787 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f938c5700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8c0c3580 con 0x7f7f8c0cf120 2022-11-27T16:04:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f91e62700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c15bd60 0x7f7f8c15e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f91e62700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c15bd60 0x7f7f8c15e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52062/0 (socket says 172.21.15.105:52062) 2022-11-27T16:04:02.788 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f91e62700 1 -- 172.21.15.105:0/384479843 learned_addr learned my addr 172.21.15.105:0/384479843 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f91661700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f8c0ae6b0 0x7f7f8c1f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.775+0000 7f7f90e60700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f8c0cf120 0x7f7f8c15c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.789 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f91e62700 1 -- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f8c0cf120 msgr2=0x7f7f8c15c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f91e62700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f8c0cf120 0x7f7f8c15c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f91e62700 1 -- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f8c0ae6b0 msgr2=0x7f7f8c1f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f91e62700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f8c0ae6b0 0x7f7f8c1f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:02.790 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f91e62700 1 -- 172.21.15.105:0/384479843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f880094a0 con 0x7f7f8c15bd60 2022-11-27T16:04:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f91e62700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c15bd60 0x7f7f8c15e740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400cc50 tx=0x7f7f8400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.791 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f79ffb700 1 -- 172.21.15.105:0/384479843 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84003c00 con 0x7f7f8c15bd60 2022-11-27T16:04:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f79ffb700 1 -- 172.21.15.105:0/384479843 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84003d60 con 0x7f7f8c15bd60 2022-11-27T16:04:02.792 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8c15ee90 con 0x7f7f8c15bd60 2022-11-27T16:04:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f79ffb700 1 -- 172.21.15.105:0/384479843 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8400aab0 con 0x7f7f8c15bd60 2022-11-27T16:04:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.776+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8c0c3210 con 0x7f7f8c15bd60 2022-11-27T16:04:02.793 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.777+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8c0ce8d0 con 0x7f7f8c15bd60 2022-11-27T16:04:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.777+0000 7f7f79ffb700 1 -- 172.21.15.105:0/384479843 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f7f84010050 con 0x7f7f8c15bd60 2022-11-27T16:04:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.777+0000 7f7f79ffb700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f7f6c066590 0x7f7f6c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.778+0000 7f7f91661700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f7f6c066590 0x7f7f6c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:02.794 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.778+0000 7f7f79ffb700 1 -- 172.21.15.105:0/384479843 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7f84083310 con 0x7f7f8c15bd60 2022-11-27T16:04:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.778+0000 7f7f91661700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f7f6c066590 0x7f7f6c068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c000f60 tx=0x7f7f7c004a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:02.795 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.782+0000 7f7f79ffb700 1 -- 172.21.15.105:0/384479843 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f7f84050d70 con 0x7f7f8c15bd60 2022-11-27T16:04:02.961 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:02.959+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7f8c067420 con 0x7f7f8c15bd60 2022-11-27T16:04:03.931 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.929+0000 7f7f79ffb700 1 -- 172.21.15.105:0/384479843 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7f84054390 con 0x7f7f8c15bd60 2022-11-27T16:04:03.935 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f7f6c066590 msgr2=0x7f7f6c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f7f6c066590 0x7f7f6c068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c000f60 tx=0x7f7f7c004a50 comp rx=0 tx=0).stop 2022-11-27T16:04:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c15bd60 msgr2=0x7f7f8c15e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:03.936 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c15bd60 0x7f7f8c15e740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7f8400cc50 tx=0x7f7f8400cf60 comp rx=0 tx=0).stop 2022-11-27T16:04:03.937 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 shutdown_connections 2022-11-27T16:04:03.937 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7f7f6c066590 0x7f7f6c068a40 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c000f60 tx=0x7f7f7c004a50 comp rx=0 tx=0).stop 2022-11-27T16:04:03.937 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f8c15bd60 0x7f7f8c15e740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:03.937 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f8c0ae6b0 0x7f7f8c1f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:03.937 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 --2- 172.21.15.105:0/384479843 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f7f8c0cf120 0x7f7f8c15c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:03.938 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.933+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 >> 172.21.15.105:0/384479843 conn(0x7f7f8c069cb0 msgr2=0x7f7f8c0ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:03.938 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.934+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 shutdown_connections 2022-11-27T16:04:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:03.934+0000 7f7f938c5700 1 -- 172.21.15.105:0/384479843 wait complete. 2022-11-27T16:04:03.939 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_metadata' created 2022-11-27T16:04:03.952 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-11-27T16:04:04.085 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.083+0000 7ff2ad39d700 1 Processor -- start 2022-11-27T16:04:04.086 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.083+0000 7ff2ad39d700 1 -- start start 2022-11-27T16:04:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2ad39d700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff2a80ceb60 0x7ff2a815f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2ad39d700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7ff2a815bb50 0x7ff2a815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:04.089 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2ad39d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a815ff60 con 0x7ff2a815b780 2022-11-27T16:04:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2ad39d700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a806f670 con 0x7ff2a80ceb60 2022-11-27T16:04:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2ad39d700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a8160480 con 0x7ff2a815bb50 2022-11-27T16:04:04.090 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2a67fc700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7ff2a80ceb60 0x7ff2a815f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2a6ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff2a815b780 0x7ff2a80ce450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43610/0 (socket says 172.21.15.105:43610) 2022-11-27T16:04:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2a6ffd700 1 -- 172.21.15.105:0/1525660830 learned_addr learned my addr 172.21.15.105:0/1525660830 (peer_addr_for_me v1:172.21.15.105:0/0) 2022-11-27T16:04:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2a67fc700 1 -- 172.21.15.105:0/1525660830 >> v2:172.21.15.170:3300/0 conn(0x7ff2a815bb50 msgr2=0x7ff2a815fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:04.091 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2a67fc700 1 --2- 172.21.15.105:0/1525660830 >> v2:172.21.15.170:3300/0 conn(0x7ff2a815bb50 0x7ff2a815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.084+0000 7ff2a67fc700 1 -- 172.21.15.105:0/1525660830 >> v1:172.21.15.105:6789/0 conn(0x7ff2a815b780 legacy=0x7ff2a80ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2a67fc700 1 -- 172.21.15.105:0/1525660830 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80c9ef0 con 0x7ff2a80ceb60 2022-11-27T16:04:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2a5ffb700 1 -- 172.21.15.105:0/1525660830 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054680562 0 0) 0x7ff29c001110 con 0x7ff2a815b780 2022-11-27T16:04:04.092 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2a67fc700 1 --2- 172.21.15.105:0/1525660830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a80ceb60 0x7ff2a815f4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80c9a60 tx=0x7ff298009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560635848b2fa4a3 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2a5ffb700 1 -- 172.21.15.105:0/1525660830 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298011830 con 0x7ff2a80ceb60 2022-11-27T16:04:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2a5ffb700 1 -- 172.21.15.105:0/1525660830 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298011990 con 0x7ff2a80ceb60 2022-11-27T16:04:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2a5ffb700 1 -- 172.21.15.105:0/1525660830 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298011d20 con 0x7ff2a80ceb60 2022-11-27T16:04:04.093 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 -- 172.21.15.105:0/1525660830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a80ceb60 msgr2=0x7ff2a815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/1525660830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a80ceb60 0x7ff2a815f4e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80c9a60 tx=0x7ff298009b20 comp rx=0 tx=0).stop 2022-11-27T16:04:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 -- 172.21.15.105:0/1525660830 shutdown_connections 2022-11-27T16:04:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/1525660830 >> v2:172.21.15.170:3300/0 conn(0x7ff2a815bb50 0x7ff2a815fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/1525660830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a80ceb60 0x7ff2a815f4e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 -- 172.21.15.105:0/1525660830 >> 172.21.15.105:0/1525660830 conn(0x7ff2a80b0080 msgr2=0x7ff2a80b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 -- 172.21.15.105:0/1525660830 shutdown_connections 2022-11-27T16:04:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.085+0000 7ff2ad39d700 1 -- 172.21.15.105:0/1525660830 wait complete. 2022-11-27T16:04:04.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.087+0000 7ff2ad39d700 1 Processor -- start 2022-11-27T16:04:04.096 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.087+0000 7ff2ad39d700 1 -- start start 2022-11-27T16:04:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2ad39d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a80ceb60 0x7ff2a81f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2ad39d700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a815b780 0x7ff2a80c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2ad39d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a815bb50 0x7ff2a80aeba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2ad39d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80c7210 con 0x7ff2a815bb50 2022-11-27T16:04:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2ad39d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80ad5f0 con 0x7ff2a80ceb60 2022-11-27T16:04:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2ad39d700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80ad8d0 con 0x7ff2a815b780 2022-11-27T16:04:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a815bb50 0x7ff2a80aeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a815bb50 0x7ff2a80aeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52080/0 (socket says 172.21.15.105:52080) 2022-11-27T16:04:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 -- 172.21.15.105:0/3079528721 learned_addr learned my addr 172.21.15.105:0/3079528721 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a67fc700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a815b780 0x7ff2a80c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a6ffd700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a80ceb60 0x7ff2a81f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:04.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 -- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a815b780 msgr2=0x7ff2a80c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a815b780 0x7ff2a80c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 -- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a80ceb60 msgr2=0x7ff2a81f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a80ceb60 0x7ff2a81f1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.088+0000 7ff2a77fe700 1 -- 172.21.15.105:0/3079528721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2980094a0 con 0x7ff2a815bb50 2022-11-27T16:04:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.089+0000 7ff2a77fe700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a815bb50 0x7ff2a80aeba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000b9e0 tx=0x7ff2a000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.089+0000 7ff28f7fe700 1 -- 172.21.15.105:0/3079528721 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000c9b0 con 0x7ff2a815bb50 2022-11-27T16:04:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.089+0000 7ff28f7fe700 1 -- 172.21.15.105:0/3079528721 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000cb50 con 0x7ff2a815bb50 2022-11-27T16:04:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.089+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80c7460 con 0x7ff2a815bb50 2022-11-27T16:04:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.089+0000 7ff28f7fe700 1 -- 172.21.15.105:0/3079528721 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0014440 con 0x7ff2a815bb50 2022-11-27T16:04:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.089+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a81f1d80 con 0x7ff2a815bb50 2022-11-27T16:04:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.090+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80c7980 con 0x7ff2a815bb50 2022-11-27T16:04:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.090+0000 7ff28f7fe700 1 -- 172.21.15.105:0/3079528721 <== mon.0 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7ff2a001b030 con 0x7ff2a815bb50 2022-11-27T16:04:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.090+0000 7ff28f7fe700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7ff288066590 0x7ff288068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.091+0000 7ff2a6ffd700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7ff288066590 0x7ff288068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:04.106 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.091+0000 7ff28f7fe700 1 -- 172.21.15.105:0/3079528721 <== mon.0 v2:172.21.15.105:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5576+0+0 (secure 0 0 0) 0x7ff2a00830d0 con 0x7ff2a815bb50 2022-11-27T16:04:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.091+0000 7ff2a6ffd700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7ff288066590 0x7ff288068a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff29c000f60 tx=0x7ff29c007a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:04.107 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.095+0000 7ff28f7fe700 1 -- 172.21.15.105:0/3079528721 <== mon.0 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7ff2a004cbd0 con 0x7ff2a815bb50 2022-11-27T16:04:04.274 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.272+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff2a8067420 con 0x7ff2a815bb50 2022-11-27T16:04:04.935 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.933+0000 7ff28f7fe700 1 -- 172.21.15.105:0/3079528721 <== mon.0 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff2a00190d0 con 0x7ff2a815bb50 2022-11-27T16:04:04.940 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7ff288066590 msgr2=0x7ff288068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:04.941 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7ff288066590 0x7ff288068a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff29c000f60 tx=0x7ff29c007a40 comp rx=0 tx=0).stop 2022-11-27T16:04:04.941 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a815bb50 msgr2=0x7ff2a80aeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:04.942 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a815bb50 0x7ff2a80aeba0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000b9e0 tx=0x7ff2a000bda0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.942 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 shutdown_connections 2022-11-27T16:04:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:6832/111926,v1:172.21.15.105:6833/111926] conn(0x7ff288066590 0x7ff288068a40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff2a815bb50 0x7ff2a80aeba0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.943 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff2a80ceb60 0x7ff2a81f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 --2- 172.21.15.105:0/3079528721 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7ff2a815b780 0x7ff2a80c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:04.944 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.938+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 >> 172.21.15.105:0/3079528721 conn(0x7ff2a80b0080 msgr2=0x7ff2a80cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:04.945 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.939+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 shutdown_connections 2022-11-27T16:04:04.946 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:04.940+0000 7ff2ad39d700 1 -- 172.21.15.105:0/3079528721 wait complete. 2022-11-27T16:04:04.948 INFO:teuthology.orchestra.run.smithi105.stderr:pool 'cephfs_data' created 2022-11-27T16:04:04.963 DEBUG:teuthology.orchestra.run.smithi105:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-11-27T16:04:05.088 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.085+0000 7f70d79bd700 1 Processor -- start 2022-11-27T16:04:05.088 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.085+0000 7f70d79bd700 1 -- start start 2022-11-27T16:04:05.093 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d79bd700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d015b9d0 0x7f70d015fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d79bd700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70d00cc940 0x7f70d0071f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d79bd700 1 --2- >> v2:172.21.15.170:3300/0 conn(0x7f70d00ae3f0 0x7f70d00724a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d79bd700 1 -- --> v2:172.21.15.105:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d015bda0 con 0x7f70d00cc940 2022-11-27T16:04:05.094 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d79bd700 1 -- --> v2:172.21.15.170:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d015f7f0 con 0x7f70d00ae3f0 2022-11-27T16:04:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d79bd700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00cceb0 con 0x7f70d015b9d0 2022-11-27T16:04:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d4f58700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70d00cc940 0x7f70d0071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d4f58700 1 --2- >> v2:172.21.15.105:3300/0 conn(0x7f70d00cc940 0x7f70d0071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52082/0 (socket says 172.21.15.105:52082) 2022-11-27T16:04:05.095 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.086+0000 7f70d4f58700 1 -- 172.21.15.105:0/3735741448 learned_addr learned my addr 172.21.15.105:0/3735741448 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d5759700 1 --2- 172.21.15.105:0/3735741448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d015b9d0 0x7f70d015fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d5f5a700 1 --2- 172.21.15.105:0/3735741448 >> v2:172.21.15.170:3300/0 conn(0x7f70d00ae3f0 0x7f70d00724a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d4f58700 1 -- 172.21.15.105:0/3735741448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d015b9d0 msgr2=0x7f70d015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:05.096 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d4f58700 1 --2- 172.21.15.105:0/3735741448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d015b9d0 0x7f70d015fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d4f58700 1 -- 172.21.15.105:0/3735741448 >> v2:172.21.15.170:3300/0 conn(0x7f70d00ae3f0 msgr2=0x7f70d00724a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:04:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d4f58700 1 --2- 172.21.15.105:0/3735741448 >> v2:172.21.15.170:3300/0 conn(0x7f70d00ae3f0 0x7f70d00724a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d4f58700 1 -- 172.21.15.105:0/3735741448 --> v2:172.21.15.105:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00ce2f0 con 0x7f70d00cc940 2022-11-27T16:04:05.097 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70d4f58700 1 --2- 172.21.15.105:0/3735741448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70d00cc940 0x7f70d0071f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b4fa0 tx=0x7f70cc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b3278fbb0992a5 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:04:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70bffff700 1 -- 172.21.15.105:0/3735741448 <== mon.0 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00b870 con 0x7f70d00cc940 2022-11-27T16:04:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70bffff700 1 -- 172.21.15.105:0/3735741448 <== mon.0 v2:172.21.15.105:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc00b9d0 con 0x7f70d00cc940 2022-11-27T16:04:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.087+0000 7f70bffff700 1 -- 172.21.15.105:0/3735741448 <== mon.0 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc00bd60 con 0x7f70d00cc940 2022-11-27T16:04:05.098 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 -- 172.21.15.105:0/3735741448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70d00cc940 msgr2=0x7f70d0071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:05.099 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 --2- 172.21.15.105:0/3735741448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70d00cc940 0x7f70d0071f60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f70d00b4fa0 tx=0x7f70cc009bd0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.099 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 -- 172.21.15.105:0/3735741448 shutdown_connections 2022-11-27T16:04:05.099 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 --2- 172.21.15.105:0/3735741448 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d015b9d0 0x7f70d015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.099 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 --2- 172.21.15.105:0/3735741448 >> v2:172.21.15.170:3300/0 conn(0x7f70d00ae3f0 0x7f70d00724a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 --2- 172.21.15.105:0/3735741448 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70d00cc940 0x7f70d0071f60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 -- 172.21.15.105:0/3735741448 >> 172.21.15.105:0/3735741448 conn(0x7f70d00cb7f0 msgr2=0x7f70d00cbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:04:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 -- 172.21.15.105:0/3735741448 shutdown_connections 2022-11-27T16:04:05.100 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.088+0000 7f70d79bd700 1 -- 172.21.15.105:0/3735741448 wait complete. 2022-11-27T16:04:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.089+0000 7f70d79bd700 1 Processor -- start 2022-11-27T16:04:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.089+0000 7f70d79bd700 1 -- start start 2022-11-27T16:04:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d79bd700 1 --2- >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d00ae3f0 0x7f70d014f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:05.101 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d79bd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d00cc940 0x7f70d00b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d79bd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70d015b9d0 0x7f70d00c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:04:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d79bd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00c6e90 con 0x7f70d015b9d0 2022-11-27T16:04:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d79bd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00c7170 con 0x7f70d00cc940 2022-11-27T16:04:05.102 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d79bd700 1 -- --> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00c7450 con 0x7f70d00ae3f0 2022-11-27T16:04:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70d015b9d0 0x7f70d00c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70d015b9d0 0x7f70d00c9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:52088/0 (socket says 172.21.15.105:52088) 2022-11-27T16:04:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 -- 172.21.15.105:0/4170776863 learned_addr learned my addr 172.21.15.105:0/4170776863 (peer_addr_for_me v2:172.21.15.105:0/0) 2022-11-27T16:04:05.103 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5759700 1 --2- 172.21.15.105:0/4170776863 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d00ae3f0 0x7f70d014f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d4f58700 1 --2- 172.21.15.105:0/4170776863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d00cc940 0x7f70d00b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:04:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 -- 172.21.15.105:0/4170776863 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d00ae3f0 msgr2=0x7f70d014f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 --2- 172.21.15.105:0/4170776863 >> [v2:172.21.15.170:3301/0,v1:172.21.15.170:6790/0] conn(0x7f70d00ae3f0 0x7f70d014f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 -- 172.21.15.105:0/4170776863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d00cc940 msgr2=0x7f70d00b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:04:05.104 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 --2- 172.21.15.105:0/4170776863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d00cc940 0x7f70d00b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:04:05.105 INFO:teuthology.orchestra.run.smithi105.stderr:2022-11-27T16:04:05.090+0000 7f70d5f5a700 1 -- 172.21.15.105:0/4170776863 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc009550 con 0x7f70d015b9d0 2022-11-27T16:04:05.105 INFO:teuthology.orchestra.run.smithi105.