2022-11-27T16:03:33.080 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T16:03:33.082 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088642 --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi --verbose -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088642/orig.config.yaml 2022-11-27T16:03:33.156 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T16:03:33.245 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088642 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088642' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - Error loading MDS rank - missing journal object - Error recovering journal - error decoding table object - failed to read JournalPointer - Corrupt directory entry - Corrupt fnode header - corrupt sessionmap header - Corrupt dentry - Scrub error on inode - Metadata damage detected - MDS_READ_ONLY - force file system read-only sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi164.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE4l3lGZ5h24xE8pTYzNyzYudEsCeJp+6ANFj26XmwfbbTyGUCCkPBXVnC4WsYtIdJWJ+CesbUzristFq8d4svrICJ29kvk4Sqtnc1MQ3ERMchxvytrwizMNa3m/DJDfQsKWYMZDfaQtiQPONidWkPtfJUqaG3DA9oqfniz6XFSjRT8AILrg6yzWosq0wUdFGb6XKp9pWCmZrntPfZ7avWKpb29gdQJsaUOi2saj/iowMhii0hi+fKAickbk+TnHKKxfRHLK9tRXuUEqIas7sv31/kGQlk4Tg3gcpjBSAh03Ou+lUQV6ckKoLiEgmQFbTDxWW/XVwXNqJJDCOXNZKVkyp6CGX3h7TiIUl0hcS1TTPoVfl7NjHG8TpLy4ON8yKuo+jjeDoTcvpER/0ZpQeuNlOsoJtHakMFcIB2Q7XSK9U5wTSa90PtsW9i4mE2jJN8pyBptiB2wKg4zaCBefMhWXwwQjTvK0ATgCdyKP7uI092Di8mkU73uMlI5xbR5YM= smithi167.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCi7ulleg3cczC64eNJF3widiQB3AdECOlLOwZ1Q9YPG17ivA3eiSfOV+bvZURS2laJZzijIYgU9q7aNOkso89rp/6q4R2QdFFelHCRuOVWZfTDjoR66oQXlHYloo0ZtwcH0za5MAYWEY6Krqgq20qqJ8TAiSWdTXSg1ZbS+AmWi8uYYM91HIPukS/ch9RUGOaHWp+oHzRIHKYI+sNftvLuS4N7PdO3w9nq4HkgBJOsv/9WUFIWlpWBQph25aVkZDWddMdLTQFPK2zeF3ep3TaNg2Uwv7I1Z5zhpjkX2BwV6bkL78d6V38rPUuvCCcBqqHM8gIsnQ6+OsnJiwXROPTH1Syb44O9836AkCEiIHoReQXw2GhuPCqNTMMicuqxF1RhHYnge4fB/ZpqQraVIZo03XYAAK+JN1Ir8lWL9VzFGIz4GT0NaTZVwHd/dbes2XOeZ04sEkBgVZFXMbDBjJoCkevZOFCBiGclvpWAdTZSNKWczvpzroVvRtF/y8DYjdc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_damage teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23126 2022-11-27T16:03:33.246 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T16:03:33.247 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T16:03:33.247 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T16:03:33.250 INFO:teuthology.task.internal:Checking packages... 2022-11-27T16:03:33.278 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T16:03:33.278 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:03:33.279 INFO:teuthology.packaging:ref: None 2022-11-27T16:03:33.280 INFO:teuthology.packaging:tag: None 2022-11-27T16:03:33.280 INFO:teuthology.packaging:branch: main 2022-11-27T16:03:33.281 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:03:33.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T16:03:33.520 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-982.g2bb1aa26 2022-11-27T16:03:33.521 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T16:03:33.531 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T16:03:33.532 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T16:03:33.573 INFO:teuthology.task.internal:Saving configuration 2022-11-27T16:03:33.593 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T16:03:33.599 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T16:03:33.624 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi164.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088642', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 15:58:18.051336', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLy+iStwE/Ya2pWTuRPj59IffuzhbBTQoRzDr2AOOg0yNig4nuJNakANiDk0ywmz6xaYY/hn+T/VWgnKdvjBrlI='} 2022-11-27T16:03:33.648 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi167.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088642', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 15:58:18.049517', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCcTSbO50mKTMBilc1osrWBdkN2IUxos2916/xKD3IP/Cwcp7SP4qG8rfbCrg/o6QUZBVxGjTw41dOTFM72P3Ok='} 2022-11-27T16:03:33.648 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T16:03:33.655 INFO:teuthology.task.internal:roles: ubuntu@smithi164.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-11-27T16:03:33.655 INFO:teuthology.task.internal:roles: ubuntu@smithi167.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-11-27T16:03:33.655 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T16:03:33.760 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f380aebe840>, signals=[15]) 2022-11-27T16:03:33.762 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T16:03:33.771 INFO:teuthology.task.internal:Opening connections... 2022-11-27T16:03:33.772 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi164.front.sepia.ceph.com 2022-11-27T16:03:33.775 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:03:33.873 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi167.front.sepia.ceph.com 2022-11-27T16:03:33.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:03:33.952 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T16:03:33.959 DEBUG:teuthology.orchestra.run.smithi164:> uname -m 2022-11-27T16:03:33.981 INFO:teuthology.orchestra.run.smithi164.stdout:x86_64 2022-11-27T16:03:33.982 DEBUG:teuthology.orchestra.run.smithi164:> cat /etc/os-release 2022-11-27T16:03:34.041 INFO:teuthology.orchestra.run.smithi164.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T16:03:34.041 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T16:03:34.041 INFO:teuthology.orchestra.run.smithi164.stdout:ID="rhel" 2022-11-27T16:03:34.041 INFO:teuthology.orchestra.run.smithi164.stdout:ID_LIKE="fedora" 2022-11-27T16:03:34.042 INFO:teuthology.orchestra.run.smithi164.stdout:VERSION_ID="8.6" 2022-11-27T16:03:34.042 INFO:teuthology.orchestra.run.smithi164.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:03:34.042 INFO:teuthology.orchestra.run.smithi164.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T16:03:34.042 INFO:teuthology.orchestra.run.smithi164.stdout:ANSI_COLOR="0;31" 2022-11-27T16:03:34.043 INFO:teuthology.orchestra.run.smithi164.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T16:03:34.043 INFO:teuthology.orchestra.run.smithi164.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T16:03:34.043 INFO:teuthology.orchestra.run.smithi164.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T16:03:34.043 INFO:teuthology.orchestra.run.smithi164.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:03:34.044 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:03:34.044 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:03:34.044 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T16:03:34.044 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T16:03:34.044 INFO:teuthology.orchestra.run.smithi164.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T16:03:34.045 INFO:teuthology.lock.ops:Updating smithi164.front.sepia.ceph.com on lock server 2022-11-27T16:03:34.074 DEBUG:teuthology.orchestra.run.smithi167:> uname -m 2022-11-27T16:03:34.094 INFO:teuthology.orchestra.run.smithi167.stdout:x86_64 2022-11-27T16:03:34.094 DEBUG:teuthology.orchestra.run.smithi167:> cat /etc/os-release 2022-11-27T16:03:34.153 INFO:teuthology.orchestra.run.smithi167.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T16:03:34.153 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T16:03:34.154 INFO:teuthology.orchestra.run.smithi167.stdout:ID="rhel" 2022-11-27T16:03:34.154 INFO:teuthology.orchestra.run.smithi167.stdout:ID_LIKE="fedora" 2022-11-27T16:03:34.154 INFO:teuthology.orchestra.run.smithi167.stdout:VERSION_ID="8.6" 2022-11-27T16:03:34.154 INFO:teuthology.orchestra.run.smithi167.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:03:34.154 INFO:teuthology.orchestra.run.smithi167.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T16:03:34.155 INFO:teuthology.orchestra.run.smithi167.stdout:ANSI_COLOR="0;31" 2022-11-27T16:03:34.155 INFO:teuthology.orchestra.run.smithi167.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T16:03:34.155 INFO:teuthology.orchestra.run.smithi167.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T16:03:34.155 INFO:teuthology.orchestra.run.smithi167.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T16:03:34.156 INFO:teuthology.orchestra.run.smithi167.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:03:34.156 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:03:34.156 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:03:34.156 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T16:03:34.156 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T16:03:34.157 INFO:teuthology.orchestra.run.smithi167.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T16:03:34.157 INFO:teuthology.lock.ops:Updating smithi167.front.sepia.ceph.com on lock server 2022-11-27T16:03:34.176 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T16:03:34.187 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T16:03:34.207 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T16:03:34.208 DEBUG:teuthology.orchestra.run.smithi164:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:03:34.211 DEBUG:teuthology.orchestra.run.smithi167:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:03:34.229 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T16:03:34.238 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T16:03:34.239 DEBUG:teuthology.orchestra.run.smithi164:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:03:34.270 DEBUG:teuthology.orchestra.run.smithi167:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:03:34.302 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T16:03:34.358 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T16:03:34.372 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2022-11-27T16:03:34.373 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2022-11-27T16:03:34.374 DEBUG:teuthology.orchestra.run.smithi164:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:03:34.375 DEBUG:teuthology.orchestra.run.smithi167:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:03:34.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:03:34.393 DEBUG:teuthology.orchestra.run.smithi167:> uname -r 2022-11-27T16:03:34.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:03:34.395 DEBUG:teuthology.orchestra.run.smithi164:> uname -r 2022-11-27T16:03:34.459 INFO:teuthology.orchestra.run.smithi167.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T16:03:34.459 INFO:teuthology.task.kernel:Running kernel on smithi167: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:03:34.460 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum install -y kernel 2022-11-27T16:03:34.462 INFO:teuthology.orchestra.run.smithi164.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T16:03:34.463 INFO:teuthology.task.kernel:Running kernel on smithi164: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:03:34.463 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum install -y kernel 2022-11-27T16:03:35.440 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2022-11-27T16:03:35.441 INFO:teuthology.orchestra.run.smithi164.stdout:Unable to read consumer identity 2022-11-27T16:03:35.448 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2022-11-27T16:03:35.449 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2022-11-27T16:03:35.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:03:35.533 INFO:teuthology.orchestra.run.smithi164.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:03:35.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:03:35.568 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:03:35.569 INFO:teuthology.orchestra.run.smithi167.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:03:35.569 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:03:39.002 INFO:teuthology.orchestra.run.smithi164.stdout:Copr repo for python3-asyncssh owned by ceph 1.5 kB/s | 3.5 kB 00:02 2022-11-27T16:03:39.207 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T16:03:40.328 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 10 MB/s | 13 MB 00:01 2022-11-27T16:03:40.657 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 13 MB 00:01 2022-11-27T16:03:43.270 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 588 kB/s | 24 kB 00:00 2022-11-27T16:03:43.633 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 423 kB/s | 24 kB 00:00 2022-11-27T16:03:43.939 INFO:teuthology.orchestra.run.smithi164.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T16:03:43.995 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2022-11-27T16:03:43.996 INFO:teuthology.orchestra.run.smithi164.stdout:Nothing to do. 2022-11-27T16:03:43.996 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2022-11-27T16:03:44.107 DEBUG:teuthology.orchestra.run.smithi164:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:03:44.327 INFO:teuthology.orchestra.run.smithi167.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T16:03:44.379 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2022-11-27T16:03:44.380 INFO:teuthology.orchestra.run.smithi167.stdout:Nothing to do. 2022-11-27T16:03:44.381 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2022-11-27T16:03:44.438 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2022-11-27T16:03:44.439 INFO:teuthology.orchestra.run.smithi164.stdout:Unable to read consumer identity 2022-11-27T16:03:44.494 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:03:44.495 INFO:teuthology.orchestra.run.smithi164.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:03:44.495 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:03:44.529 DEBUG:teuthology.orchestra.run.smithi167:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:03:44.732 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Nov 2022 04:03:43 PM UTC. 2022-11-27T16:03:44.774 INFO:teuthology.orchestra.run.smithi164.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:03:44.813 INFO:teuthology.orchestra.run.smithi164.stderr:Error: No packages marked for reinstall. 2022-11-27T16:03:44.860 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum reinstall -y kernel || true 2022-11-27T16:03:44.861 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2022-11-27T16:03:44.861 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2022-11-27T16:03:44.901 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:03:44.901 INFO:teuthology.orchestra.run.smithi167.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:03:44.902 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:03:45.149 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 04:03:43 PM UTC. 2022-11-27T16:03:45.187 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2022-11-27T16:03:45.187 INFO:teuthology.orchestra.run.smithi164.stdout:Unable to read consumer identity 2022-11-27T16:03:45.195 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:03:45.230 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:03:45.230 INFO:teuthology.orchestra.run.smithi164.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:03:45.230 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:03:45.243 INFO:teuthology.orchestra.run.smithi167.stderr:Error: No packages marked for reinstall. 2022-11-27T16:03:45.289 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum reinstall -y kernel || true 2022-11-27T16:03:45.468 INFO:teuthology.orchestra.run.smithi164.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 04:03:43 PM UTC. 2022-11-27T16:03:45.506 INFO:teuthology.orchestra.run.smithi164.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:03:45.535 INFO:teuthology.orchestra.run.smithi164.stderr:Error: No packages marked for reinstall. 2022-11-27T16:03:45.586 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q kernel --last | head -n 1 2022-11-27T16:03:45.622 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2022-11-27T16:03:45.623 INFO:teuthology.orchestra.run.smithi167.stdout:Unable to read consumer identity 2022-11-27T16:03:45.666 INFO:teuthology.orchestra.run.smithi164.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T16:03:45.668 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:03:45.669 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:03:45.669 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2022-11-27T16:03:45.669 DEBUG:teuthology.orchestra.run.smithi164:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T16:03:45.676 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:03:45.676 INFO:teuthology.orchestra.run.smithi167.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:03:45.676 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:03:45.709 INFO:teuthology.orchestra.run.smithi164.stdout:ttyS1 2022-11-27T16:03:45.724 DEBUG:teuthology.parallel:result is None 2022-11-27T16:03:45.920 INFO:teuthology.orchestra.run.smithi167.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 04:03:43 PM UTC. 2022-11-27T16:03:45.964 INFO:teuthology.orchestra.run.smithi167.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:03:45.995 INFO:teuthology.orchestra.run.smithi167.stderr:Error: No packages marked for reinstall. 2022-11-27T16:03:46.047 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q kernel --last | head -n 1 2022-11-27T16:03:46.127 INFO:teuthology.orchestra.run.smithi167.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T16:03:46.129 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:03:46.129 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:03:46.130 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2022-11-27T16:03:46.130 DEBUG:teuthology.orchestra.run.smithi167:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2022-11-27T16:03:46.164 INFO:teuthology.orchestra.run.smithi167.stdout:ttyS1 2022-11-27T16:03:46.180 DEBUG:teuthology.parallel:result is None 2022-11-27T16:03:46.181 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T16:03:46.189 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T16:03:46.190 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:03:46.192 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:03:46.238 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T16:03:46.245 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T16:03:46.289 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T16:03:46.290 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:03:46.292 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:03:46.323 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T16:03:46.330 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T16:03:46.331 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:03:46.361 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:03:46.394 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:03:46.398 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:03:46.404 INFO:teuthology.orchestra.run.smithi167.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:03:46.409 INFO:teuthology.orchestra.run.smithi164.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:03:46.411 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T16:03:46.418 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T16:03:46.419 DEBUG:teuthology.orchestra.run.smithi164:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:03:46.454 DEBUG:teuthology.orchestra.run.smithi167:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:03:46.483 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T16:03:46.496 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T16:03:46.497 DEBUG:teuthology.orchestra.run.smithi164:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:03:46.521 DEBUG:teuthology.orchestra.run.smithi167:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:03:46.541 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:03:46.623 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:03:46.685 DEBUG:teuthology.orchestra.run.smithi164:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:03:46.733 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:03:46.771 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:03:46.772 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:03:46.838 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:03:46.885 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:03:46.940 DEBUG:teuthology.orchestra.run.smithi167:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:03:47.025 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:03:47.066 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:03:47.066 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:03:47.134 DEBUG:teuthology.orchestra.run.smithi164:> sudo service rsyslog restart 2022-11-27T16:03:47.138 DEBUG:teuthology.orchestra.run.smithi167:> sudo service rsyslog restart 2022-11-27T16:03:47.187 INFO:teuthology.orchestra.run.smithi164.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:03:47.232 INFO:teuthology.orchestra.run.smithi167.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:03:47.610 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T16:03:47.623 INFO:teuthology.task.internal:Starting timer... 2022-11-27T16:03:47.624 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T16:03:47.655 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T16:03:47.695 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T16:03:47.696 DEBUG:teuthology.orchestra.run.smithi164:> sudo service auditd rotate 2022-11-27T16:03:47.752 INFO:teuthology.orchestra.run.smithi164.stdout:Rotating logs: 2022-11-27T16:03:47.754 DEBUG:teuthology.orchestra.run.smithi167:> sudo service auditd rotate 2022-11-27T16:03:47.810 INFO:teuthology.orchestra.run.smithi167.stdout:Rotating logs: 2022-11-27T16:03:47.812 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T16:03:47.813 DEBUG:teuthology.orchestra.run.smithi164:> /usr/sbin/getenforce 2022-11-27T16:03:47.837 INFO:teuthology.orchestra.run.smithi164.stdout:Permissive 2022-11-27T16:03:47.838 DEBUG:teuthology.orchestra.run.smithi167:> /usr/sbin/getenforce 2022-11-27T16:03:47.864 INFO:teuthology.orchestra.run.smithi167.stdout:Permissive 2022-11-27T16:03:47.865 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi164.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi167.front.sepia.ceph.com': 'permissive'} 2022-11-27T16:03:47.865 DEBUG:teuthology.orchestra.run.smithi164:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:03:47.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:03:47.906 DEBUG:teuthology.orchestra.run.smithi167:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:03:47.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:03:47.934 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T16:03:47.934 DEBUG:teuthology.orchestra.run.smithi164:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:03:47.994 DEBUG:teuthology.orchestra.run.smithi167:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:03:48.024 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T16:03:48.038 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2022-11-27T16:03:48.053 INFO:teuthology.repo_utils:Fetching main from origin 2022-11-27T16:03:48.203 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T16:03:48.232 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T16:03:48.235 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi164.front.sepia.ceph.com,smithi167.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T16:11:50.787 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi164.front.sepia.ceph.com'), Remote(name='ubuntu@smithi167.front.sepia.ceph.com')] 2022-11-27T16:11:50.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi164.front.sepia.ceph.com' 2022-11-27T16:11:50.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi164.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:11:50.895 DEBUG:teuthology.orchestra.run.smithi164:> true 2022-11-27T16:11:50.961 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi167.front.sepia.ceph.com' 2022-11-27T16:11:50.962 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi167.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:11:51.048 DEBUG:teuthology.orchestra.run.smithi167:> true 2022-11-27T16:11:51.118 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T16:11:51.131 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T16:11:51.131 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:11:51.134 DEBUG:teuthology.orchestra.run.smithi164:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:11:51.137 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:11:51.138 DEBUG:teuthology.orchestra.run.smithi167:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:11:51.167 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:11:51.181 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:11:51.190 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:11:51.205 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:11:51.246 INFO:teuthology.orchestra.run.smithi164.stderr:sudo: ntpd: command not found 2022-11-27T16:11:51.258 INFO:teuthology.orchestra.run.smithi164.stdout:506 Cannot talk to daemon 2022-11-27T16:11:51.272 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:11:51.278 INFO:teuthology.orchestra.run.smithi167.stderr:sudo: ntpd: command not found 2022-11-27T16:11:51.286 INFO:teuthology.orchestra.run.smithi164.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:11:51.291 INFO:teuthology.orchestra.run.smithi167.stdout:506 Cannot talk to daemon 2022-11-27T16:11:51.306 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:11:51.320 INFO:teuthology.orchestra.run.smithi167.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:11:51.359 INFO:teuthology.orchestra.run.smithi164.stderr:bash: ntpq: command not found 2022-11-27T16:11:51.362 INFO:teuthology.orchestra.run.smithi164.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:11:51.362 INFO:teuthology.orchestra.run.smithi164.stdout:=============================================================================== 2022-11-27T16:11:51.363 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.363 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.363 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.364 INFO:teuthology.orchestra.run.smithi164.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.390 INFO:teuthology.orchestra.run.smithi167.stderr:bash: ntpq: command not found 2022-11-27T16:11:51.393 INFO:teuthology.orchestra.run.smithi167.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:11:51.394 INFO:teuthology.orchestra.run.smithi167.stdout:=============================================================================== 2022-11-27T16:11:51.394 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.394 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.394 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.395 INFO:teuthology.orchestra.run.smithi167.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:11:51.396 INFO:teuthology.run_tasks:Running task install... 2022-11-27T16:11:51.406 DEBUG:teuthology.task.install:project ceph 2022-11-27T16:11:51.407 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T16:11:51.407 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T16:11:51.407 INFO:teuthology.task.install:Using flavor: default 2022-11-27T16:11:51.414 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T16:11:51.415 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-11-27T16:11:51.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo lsb_release -is 2022-11-27T16:11:51.457 INFO:teuthology.orchestra.run.smithi164.stdout:RedHatEnterprise 2022-11-27T16:11:51.458 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:11:51.459 DEBUG:teuthology.orchestra.run.smithi167:> sudo lsb_release -is 2022-11-27T16:11:51.460 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:11:51.461 DEBUG:teuthology.orchestra.run.smithi164:> sudo lsb_release -is 2022-11-27T16:11:51.502 INFO:teuthology.orchestra.run.smithi167.stdout:RedHatEnterprise 2022-11-27T16:11:51.504 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:11:51.504 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:11:51.505 DEBUG:teuthology.orchestra.run.smithi167:> sudo lsb_release -is 2022-11-27T16:11:51.540 INFO:teuthology.orchestra.run.smithi164.stdout:RedHatEnterprise 2022-11-27T16:11:51.541 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:11:51.542 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:11:51.587 INFO:teuthology.orchestra.run.smithi167.stdout:RedHatEnterprise 2022-11-27T16:11:51.588 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:11:51.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:11:51.705 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:11:51.707 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:11:51.758 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:11:51.759 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:11:51.836 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:11:51.837 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:11:51.837 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:11:51.869 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:11:51.870 DEBUG:teuthology.orchestra.run.smithi164:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:11:51.886 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:11:51.886 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:11:51.887 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:11:51.916 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:11:51.917 DEBUG:teuthology.orchestra.run.smithi167:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:11:51.944 DEBUG:teuthology.orchestra.run.smithi164:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:11:51.989 DEBUG:teuthology.orchestra.run.smithi167:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:11:52.026 DEBUG:teuthology.orchestra.run.smithi164:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:11:52.077 DEBUG:teuthology.orchestra.run.smithi167:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:11:52.101 INFO:teuthology.orchestra.run.smithi164.stdout:check_obsoletes = 1 2022-11-27T16:11:52.103 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum clean all 2022-11-27T16:11:52.144 INFO:teuthology.orchestra.run.smithi167.stdout:check_obsoletes = 1 2022-11-27T16:11:52.146 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum clean all 2022-11-27T16:11:52.475 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2022-11-27T16:11:52.516 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2022-11-27T16:11:52.712 INFO:teuthology.orchestra.run.smithi164.stdout:54 files removed 2022-11-27T16:11:52.739 INFO:teuthology.orchestra.run.smithi167.stdout:54 files removed 2022-11-27T16:11:52.759 DEBUG:teuthology.orchestra.run.smithi164:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:11:52.787 DEBUG:teuthology.orchestra.run.smithi167:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:11:53.093 INFO:teuthology.orchestra.run.smithi164.stdout:Updating Subscription Management repositories. 2022-11-27T16:11:53.117 INFO:teuthology.orchestra.run.smithi167.stdout:Updating Subscription Management repositories. 2022-11-27T16:11:53.749 INFO:teuthology.orchestra.run.smithi164.stdout:ceph packages for x86_64 295 kB/s | 81 kB 00:00 2022-11-27T16:11:53.787 INFO:teuthology.orchestra.run.smithi167.stdout:ceph packages for x86_64 273 kB/s | 81 kB 00:00 2022-11-27T16:11:54.034 INFO:teuthology.orchestra.run.smithi164.stdout:ceph noarch packages 37 kB/s | 9.4 kB 00:00 2022-11-27T16:11:54.088 INFO:teuthology.orchestra.run.smithi167.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2022-11-27T16:11:54.284 INFO:teuthology.orchestra.run.smithi164.stdout:ceph source packages 6.6 kB/s | 1.5 kB 00:00 2022-11-27T16:11:54.372 INFO:teuthology.orchestra.run.smithi167.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2022-11-27T16:11:56.277 INFO:teuthology.orchestra.run.smithi164.stdout:Copr repo for python3-asyncssh owned by ceph 1.8 kB/s | 3.5 kB 00:01 2022-11-27T16:11:57.348 INFO:teuthology.orchestra.run.smithi167.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2022-11-27T16:11:57.569 INFO:teuthology.orchestra.run.smithi164.stdout:Extra Packages for Enterprise Linux 10 MB/s | 13 MB 00:01 2022-11-27T16:11:58.565 INFO:teuthology.orchestra.run.smithi167.stdout:Extra Packages for Enterprise Linux 11 MB/s | 13 MB 00:01 2022-11-27T16:12:00.510 INFO:teuthology.orchestra.run.smithi164.stdout:lab-extras 403 kB/s | 24 kB 00:00 2022-11-27T16:12:01.321 INFO:teuthology.orchestra.run.smithi164.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 64 MB/s | 48 MB 00:00 2022-11-27T16:12:01.556 INFO:teuthology.orchestra.run.smithi167.stdout:lab-extras 314 kB/s | 24 kB 00:00 2022-11-27T16:12:02.384 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 70 MB/s | 53 MB 00:00 2022-11-27T16:12:09.446 INFO:teuthology.orchestra.run.smithi164.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 69 MB/s | 53 MB 00:00 2022-11-27T16:12:10.377 INFO:teuthology.orchestra.run.smithi167.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 64 MB/s | 48 MB 00:00 2022-11-27T16:12:20.494 INFO:teuthology.orchestra.run.smithi164.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:12:20.495 INFO:teuthology.orchestra.run.smithi164.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:12:20.510 INFO:teuthology.orchestra.run.smithi164.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T16:12:20.511 INFO:teuthology.orchestra.run.smithi164.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T16:12:20.512 INFO:teuthology.orchestra.run.smithi164.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T16:12:20.513 INFO:teuthology.orchestra.run.smithi164.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:12:20.515 INFO:teuthology.orchestra.run.smithi164.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:12:20.519 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:12:20.520 INFO:teuthology.orchestra.run.smithi164.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:12:20.652 INFO:teuthology.orchestra.run.smithi164.stdout:Dependencies resolved. 2022-11-27T16:12:20.665 INFO:teuthology.orchestra.run.smithi164.stdout:=========================================================================================================================== 2022-11-27T16:12:20.665 INFO:teuthology.orchestra.run.smithi164.stdout: Package Arch Version Repository Size 2022-11-27T16:12:20.665 INFO:teuthology.orchestra.run.smithi164.stdout:=========================================================================================================================== 2022-11-27T16:12:20.665 INFO:teuthology.orchestra.run.smithi164.stdout:Installing: 2022-11-27T16:12:20.666 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T16:12:20.666 INFO:teuthology.orchestra.run.smithi164.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T16:12:20.666 INFO:teuthology.orchestra.run.smithi164.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:12:20.666 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:12:20.667 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:12:20.667 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:12:20.667 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:12:20.667 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:12:20.668 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:12:20.668 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:12:20.668 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:12:20.668 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:12:20.668 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:12:20.669 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:12:20.669 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:12:20.669 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:12:20.669 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:12:20.670 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T16:12:20.670 INFO:teuthology.orchestra.run.smithi164.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T16:12:20.670 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T16:12:20.670 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:12:20.670 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:12:20.671 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:12:20.671 INFO:teuthology.orchestra.run.smithi164.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T16:12:20.671 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T16:12:20.671 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T16:12:20.672 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T16:12:20.672 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T16:12:20.672 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T16:12:20.672 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:12:20.672 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:12:20.673 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:12:20.673 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:12:20.673 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:12:20.673 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:12:20.674 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:12:20.674 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T16:12:20.674 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrading: 2022-11-27T16:12:20.674 INFO:teuthology.orchestra.run.smithi164.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T16:12:20.674 INFO:teuthology.orchestra.run.smithi164.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:12:20.675 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:12:20.675 INFO:teuthology.orchestra.run.smithi164.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T16:12:20.675 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T16:12:20.675 INFO:teuthology.orchestra.run.smithi164.stdout:Installing dependencies: 2022-11-27T16:12:20.676 INFO:teuthology.orchestra.run.smithi164.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T16:12:20.676 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:12:20.676 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:12:20.676 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:12:20.676 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:12:20.677 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:12:20.677 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:12:20.677 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:12:20.677 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:12:20.678 INFO:teuthology.orchestra.run.smithi164.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T16:12:20.678 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:12:20.678 INFO:teuthology.orchestra.run.smithi164.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T16:12:20.678 INFO:teuthology.orchestra.run.smithi164.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T16:12:20.678 INFO:teuthology.orchestra.run.smithi164.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T16:12:20.679 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:12:20.679 INFO:teuthology.orchestra.run.smithi164.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T16:12:20.679 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T16:12:20.679 INFO:teuthology.orchestra.run.smithi164.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:12:20.680 INFO:teuthology.orchestra.run.smithi164.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T16:12:20.680 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T16:12:20.680 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T16:12:20.680 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:12:20.681 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T16:12:20.681 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:12:20.681 INFO:teuthology.orchestra.run.smithi164.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T16:12:20.681 INFO:teuthology.orchestra.run.smithi164.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T16:12:20.681 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:12:20.682 INFO:teuthology.orchestra.run.smithi164.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T16:12:20.682 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T16:12:20.682 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T16:12:20.682 INFO:teuthology.orchestra.run.smithi164.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T16:12:20.682 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T16:12:20.683 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T16:12:20.683 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T16:12:20.683 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T16:12:20.683 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:12:20.684 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T16:12:20.684 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:12:20.684 INFO:teuthology.orchestra.run.smithi164.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:12:20.684 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:12:20.684 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:12:20.685 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:12:20.685 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:12:20.685 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T16:12:20.685 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:12:20.685 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:12:20.686 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T16:12:20.686 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:12:20.686 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:12:20.686 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:12:20.687 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T16:12:20.687 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T16:12:20.687 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T16:12:20.687 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:12:20.687 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:12:20.688 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:12:20.688 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T16:12:20.688 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T16:12:20.688 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:12:20.688 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:12:20.689 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T16:12:20.689 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T16:12:20.689 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T16:12:20.689 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:12:20.690 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:12:20.690 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T16:12:20.690 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T16:12:20.690 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T16:12:20.690 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T16:12:20.691 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T16:12:20.691 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T16:12:20.691 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:12:20.691 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:12:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:12:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:12:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T16:12:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:12:20.692 INFO:teuthology.orchestra.run.smithi164.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:12:20.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:12:20.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:12:20.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:12:20.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:12:20.693 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:12:20.694 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:12:20.694 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T16:12:20.694 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:12:20.694 INFO:teuthology.orchestra.run.smithi164.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T16:12:20.695 INFO:teuthology.orchestra.run.smithi164.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:12:20.695 INFO:teuthology.orchestra.run.smithi164.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T16:12:20.695 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T16:12:20.695 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T16:12:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:Enabling module streams: 2022-11-27T16:12:20.696 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql 10 2022-11-27T16:12:20.696 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:12:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction Summary 2022-11-27T16:12:20.696 INFO:teuthology.orchestra.run.smithi164.stdout:=========================================================================================================================== 2022-11-27T16:12:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:Install 128 Packages 2022-11-27T16:12:20.697 INFO:teuthology.orchestra.run.smithi164.stdout:Upgrade 5 Packages 2022-11-27T16:12:20.697 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:12:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:Total download size: 219 M 2022-11-27T16:12:20.698 INFO:teuthology.orchestra.run.smithi164.stdout:Downloading Packages: 2022-11-27T16:12:20.813 INFO:teuthology.orchestra.run.smithi164.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2022-11-27T16:12:21.013 INFO:teuthology.orchestra.run.smithi164.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 4.1 MB/s | 845 kB 00:00 2022-11-27T16:12:21.080 INFO:teuthology.orchestra.run.smithi164.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 2.4 MB/s | 165 kB 00:00 2022-11-27T16:12:21.422 INFO:teuthology.orchestra.run.smithi164.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 7.0 MB/s | 5.2 MB 00:00 2022-11-27T16:12:21.464 INFO:teuthology.orchestra.run.smithi164.stdout:(5/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 5.9 MB/s | 2.3 MB 00:00 2022-11-27T16:12:21.484 INFO:teuthology.orchestra.run.smithi167.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:12:21.485 INFO:teuthology.orchestra.run.smithi167.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:12:21.500 INFO:teuthology.orchestra.run.smithi167.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T16:12:21.501 INFO:teuthology.orchestra.run.smithi167.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T16:12:21.502 INFO:teuthology.orchestra.run.smithi167.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T16:12:21.503 INFO:teuthology.orchestra.run.smithi167.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:12:21.506 INFO:teuthology.orchestra.run.smithi167.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:12:21.509 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:12:21.510 INFO:teuthology.orchestra.run.smithi167.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:12:21.623 INFO:teuthology.orchestra.run.smithi164.stdout:(6/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 7.7 MB/s | 1.5 MB 00:00 2022-11-27T16:12:21.643 INFO:teuthology.orchestra.run.smithi167.stdout:Dependencies resolved. 2022-11-27T16:12:21.656 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2022-11-27T16:12:21.657 INFO:teuthology.orchestra.run.smithi167.stdout: Package Arch Version Repository Size 2022-11-27T16:12:21.657 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2022-11-27T16:12:21.657 INFO:teuthology.orchestra.run.smithi167.stdout:Installing: 2022-11-27T16:12:21.658 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T16:12:21.658 INFO:teuthology.orchestra.run.smithi167.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T16:12:21.658 INFO:teuthology.orchestra.run.smithi167.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:12:21.658 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:12:21.659 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:12:21.659 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:12:21.659 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:12:21.659 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:12:21.660 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:12:21.660 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:12:21.660 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:12:21.660 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:12:21.661 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:12:21.661 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:12:21.661 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:12:21.661 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:12:21.662 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:12:21.662 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T16:12:21.662 INFO:teuthology.orchestra.run.smithi167.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T16:12:21.662 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T16:12:21.662 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:12:21.663 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:12:21.663 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:12:21.663 INFO:teuthology.orchestra.run.smithi167.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T16:12:21.663 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T16:12:21.664 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T16:12:21.664 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T16:12:21.664 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T16:12:21.664 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T16:12:21.665 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:12:21.665 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:12:21.665 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:12:21.666 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:12:21.666 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:12:21.666 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:12:21.667 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:12:21.667 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T16:12:21.667 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrading: 2022-11-27T16:12:21.667 INFO:teuthology.orchestra.run.smithi167.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T16:12:21.668 INFO:teuthology.orchestra.run.smithi167.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:12:21.668 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:12:21.668 INFO:teuthology.orchestra.run.smithi167.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T16:12:21.669 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T16:12:21.669 INFO:teuthology.orchestra.run.smithi167.stdout:Installing dependencies: 2022-11-27T16:12:21.669 INFO:teuthology.orchestra.run.smithi167.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T16:12:21.669 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:12:21.670 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:12:21.670 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:12:21.670 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:12:21.670 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:12:21.671 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:12:21.671 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:12:21.671 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:12:21.671 INFO:teuthology.orchestra.run.smithi167.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T16:12:21.672 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:12:21.672 INFO:teuthology.orchestra.run.smithi167.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T16:12:21.672 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T16:12:21.672 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T16:12:21.672 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:12:21.673 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T16:12:21.673 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T16:12:21.673 INFO:teuthology.orchestra.run.smithi167.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:12:21.673 INFO:teuthology.orchestra.run.smithi167.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T16:12:21.674 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T16:12:21.674 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T16:12:21.674 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:12:21.674 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T16:12:21.674 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:12:21.675 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T16:12:21.675 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T16:12:21.676 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:12:21.676 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T16:12:21.676 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T16:12:21.677 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T16:12:21.677 INFO:teuthology.orchestra.run.smithi167.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T16:12:21.677 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T16:12:21.677 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T16:12:21.678 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T16:12:21.678 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T16:12:21.678 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:12:21.678 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T16:12:21.679 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:12:21.679 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:12:21.680 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:12:21.680 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:12:21.680 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:12:21.680 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:12:21.680 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T16:12:21.681 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:12:21.681 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:12:21.681 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T16:12:21.681 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:12:21.682 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:12:21.682 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:12:21.682 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T16:12:21.682 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T16:12:21.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T16:12:21.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:12:21.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:12:21.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:12:21.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T16:12:21.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T16:12:21.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:12:21.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:12:21.684 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T16:12:21.685 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T16:12:21.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T16:12:21.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:12:21.686 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:12:21.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T16:12:21.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T16:12:21.687 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T16:12:21.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T16:12:21.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T16:12:21.688 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T16:12:21.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:12:21.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:12:21.689 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:12:21.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:12:21.690 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T16:12:21.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:12:21.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:12:21.691 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:12:21.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:12:21.692 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:12:21.693 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:12:21.693 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:12:21.693 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:12:21.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T16:12:21.694 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:12:21.694 INFO:teuthology.orchestra.run.smithi167.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T16:12:21.695 INFO:teuthology.orchestra.run.smithi167.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:12:21.695 INFO:teuthology.orchestra.run.smithi167.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T16:12:21.696 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T16:12:21.696 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T16:12:21.696 INFO:teuthology.orchestra.run.smithi167.stdout:Enabling module streams: 2022-11-27T16:12:21.697 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql 10 2022-11-27T16:12:21.698 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:12:21.698 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction Summary 2022-11-27T16:12:21.698 INFO:teuthology.orchestra.run.smithi167.stdout:=========================================================================================================================== 2022-11-27T16:12:21.699 INFO:teuthology.orchestra.run.smithi167.stdout:Install 128 Packages 2022-11-27T16:12:21.700 INFO:teuthology.orchestra.run.smithi167.stdout:Upgrade 5 Packages 2022-11-27T16:12:21.700 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:12:21.701 INFO:teuthology.orchestra.run.smithi167.stdout:Total download size: 219 M 2022-11-27T16:12:21.701 INFO:teuthology.orchestra.run.smithi167.stdout:Downloading Packages: 2022-11-27T16:12:21.974 INFO:teuthology.orchestra.run.smithi167.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 21 kB/s | 6.4 kB 00:00 2022-11-27T16:12:22.024 INFO:teuthology.orchestra.run.smithi164.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 6.9 MB/s | 3.9 MB 00:00 2022-11-27T16:12:23.268 INFO:teuthology.orchestra.run.smithi167.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 653 kB/s | 845 kB 00:01 2022-11-27T16:12:23.298 INFO:teuthology.orchestra.run.smithi164.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 9.3 MB/s | 24 MB 00:02 2022-11-27T16:12:23.381 INFO:teuthology.orchestra.run.smithi164.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 293 kB/s | 24 kB 00:00 2022-11-27T16:12:23.444 INFO:teuthology.orchestra.run.smithi167.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 946 kB/s | 165 kB 00:00 2022-11-27T16:12:23.707 INFO:teuthology.orchestra.run.smithi164.stdout:(10/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 8.1 MB/s | 17 MB 00:02 2022-11-27T16:12:23.799 INFO:teuthology.orchestra.run.smithi164.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 2.7 MB/s | 257 kB 00:00 2022-11-27T16:12:23.882 INFO:teuthology.orchestra.run.smithi164.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 366 kB/s | 30 kB 00:00 2022-11-27T16:12:23.911 INFO:teuthology.orchestra.run.smithi167.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 2.3 MB/s | 5.2 MB 00:02 2022-11-27T16:12:24.058 INFO:teuthology.orchestra.run.smithi164.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 4.3 MB/s | 767 kB 00:00 2022-11-27T16:12:24.149 INFO:teuthology.orchestra.run.smithi164.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 1.5 MB/s | 137 kB 00:00 2022-11-27T16:12:24.241 INFO:teuthology.orchestra.run.smithi164.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.5 MB/s | 139 kB 00:00 2022-11-27T16:12:24.341 INFO:teuthology.orchestra.run.smithi164.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 4.6 MB/s | 474 kB 00:00 2022-11-27T16:12:24.578 INFO:teuthology.orchestra.run.smithi167.stdout:(5/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 2.3 MB/s | 1.5 MB 00:00 2022-11-27T16:12:24.792 INFO:teuthology.orchestra.run.smithi164.stdout:(17/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 6.6 MB/s | 18 MB 00:02 2022-11-27T16:12:24.875 INFO:teuthology.orchestra.run.smithi164.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 536 kB/s | 45 kB 00:00 2022-11-27T16:12:24.969 INFO:teuthology.orchestra.run.smithi164.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.2 MB/s | 109 kB 00:00 2022-11-27T16:12:25.068 INFO:teuthology.orchestra.run.smithi164.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 1.9 MB/s | 190 kB 00:00 2022-11-27T16:12:25.168 INFO:teuthology.orchestra.run.smithi164.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 3.6 MB/s | 371 kB 00:00 2022-11-27T16:12:25.285 INFO:teuthology.orchestra.run.smithi164.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 3.0 MB/s | 357 kB 00:00 2022-11-27T16:12:25.393 INFO:teuthology.orchestra.run.smithi164.stdout:(23/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 6.1 MB/s | 6.5 MB 00:01 2022-11-27T16:12:25.410 INFO:teuthology.orchestra.run.smithi164.stdout:(24/133): python3-rgw-18.0.0-594.g697b0d99.el8. 851 kB/s | 107 kB 00:00 2022-11-27T16:12:25.469 INFO:teuthology.orchestra.run.smithi164.stdout:(25/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.2 MB/s | 94 kB 00:00 2022-11-27T16:12:25.536 INFO:teuthology.orchestra.run.smithi167.stdout:(6/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 1.1 MB/s | 2.3 MB 00:02 2022-11-27T16:12:25.561 INFO:teuthology.orchestra.run.smithi164.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 1.9 MB/s | 175 kB 00:00 2022-11-27T16:12:25.662 INFO:teuthology.orchestra.run.smithi164.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 228 kB/s | 23 kB 00:00 2022-11-27T16:12:25.745 INFO:teuthology.orchestra.run.smithi164.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.4 MB/s | 118 kB 00:00 2022-11-27T16:12:25.778 INFO:teuthology.orchestra.run.smithi167.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 3.2 MB/s | 3.9 MB 00:01 2022-11-27T16:12:26.079 INFO:teuthology.orchestra.run.smithi164.stdout:(29/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 5.0 MB/s | 1.7 MB 00:00 2022-11-27T16:12:26.129 INFO:teuthology.orchestra.run.smithi164.stdout:(30/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 4.8 MB/s | 3.5 MB 00:00 2022-11-27T16:12:26.229 INFO:teuthology.orchestra.run.smithi164.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.3 MB/s | 234 kB 00:00 2022-11-27T16:12:26.321 INFO:teuthology.orchestra.run.smithi164.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 512 kB/s | 47 kB 00:00 2022-11-27T16:12:26.413 INFO:teuthology.orchestra.run.smithi164.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 157 kB/s | 14 kB 00:00 2022-11-27T16:12:26.514 INFO:teuthology.orchestra.run.smithi164.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 2.6 MB/s | 263 kB 00:00 2022-11-27T16:12:26.605 INFO:teuthology.orchestra.run.smithi164.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 887 kB/s | 81 kB 00:00 2022-11-27T16:12:26.723 INFO:teuthology.orchestra.run.smithi164.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 131 kB/s | 15 kB 00:00 2022-11-27T16:12:26.922 INFO:teuthology.orchestra.run.smithi164.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2022-11-27T16:12:26.965 INFO:teuthology.orchestra.run.smithi164.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2022-11-27T16:12:27.015 INFO:teuthology.orchestra.run.smithi164.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2022-11-27T16:12:27.098 INFO:teuthology.orchestra.run.smithi164.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2022-11-27T16:12:27.149 INFO:teuthology.orchestra.run.smithi164.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 876 kB/s | 44 kB 00:00 2022-11-27T16:12:27.241 INFO:teuthology.orchestra.run.smithi164.stdout:(42/133): ceph-mgr-diskprediction-local-18.0.0- 6.4 MB/s | 7.4 MB 00:01 2022-11-27T16:12:27.257 INFO:teuthology.orchestra.run.smithi164.stdout:(43/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2022-11-27T16:12:27.308 INFO:teuthology.orchestra.run.smithi164.stdout:(44/133): python3-certifi-2018.10.15-7.el8.noar 325 kB/s | 16 kB 00:00 2022-11-27T16:12:27.375 INFO:teuthology.orchestra.run.smithi164.stdout:(45/133): python3-cachetools-3.1.1-4.el8.noarch 247 kB/s | 33 kB 00:00 2022-11-27T16:12:27.409 INFO:teuthology.orchestra.run.smithi164.stdout:(46/133): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2022-11-27T16:12:27.476 INFO:teuthology.orchestra.run.smithi164.stdout:(47/133): python3-google-auth-1.1.1-10.el8.noar 1.2 MB/s | 81 kB 00:00 2022-11-27T16:12:27.984 INFO:teuthology.orchestra.run.smithi164.stdout:(48/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 12 MB/s | 55 MB 00:04 2022-11-27T16:12:28.002 INFO:teuthology.orchestra.run.smithi164.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 21 kB/s | 11 kB 00:00 2022-11-27T16:12:28.027 INFO:teuthology.orchestra.run.smithi164.stdout:(50/133): python3-cherrypy-18.4.0-1.el8.noarch. 591 kB/s | 384 kB 00:00 2022-11-27T16:12:28.043 INFO:teuthology.orchestra.run.smithi164.stdout:(51/133): python3-jaraco-functools-2.0-4.el8.no 301 kB/s | 18 kB 00:00 2022-11-27T16:12:28.069 INFO:teuthology.orchestra.run.smithi164.stdout:(52/133): python3-logutils-0.3.5-11.el8.noarch. 1.1 MB/s | 49 kB 00:00 2022-11-27T16:12:28.086 INFO:teuthology.orchestra.run.smithi164.stdout:(53/133): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2022-11-27T16:12:28.111 INFO:teuthology.orchestra.run.smithi164.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2022-11-27T16:12:28.153 INFO:teuthology.orchestra.run.smithi164.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 375 kB/s | 16 kB 00:00 2022-11-27T16:12:28.188 INFO:teuthology.orchestra.run.smithi164.stdout:(56/133): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2022-11-27T16:12:28.204 INFO:teuthology.orchestra.run.smithi164.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 667 kB/s | 34 kB 00:00 2022-11-27T16:12:28.246 INFO:teuthology.orchestra.run.smithi164.stdout:(58/133): python3-routes-2.4.1-12.el8.noarch.rp 3.3 MB/s | 196 kB 00:00 2022-11-27T16:12:28.264 INFO:teuthology.orchestra.run.smithi164.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 1.0 MB/s | 61 kB 00:00 2022-11-27T16:12:28.287 INFO:teuthology.orchestra.run.smithi164.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 450 kB/s | 19 kB 00:00 2022-11-27T16:12:28.334 INFO:teuthology.orchestra.run.smithi164.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 333 kB/s | 24 kB 00:00 2022-11-27T16:12:28.363 INFO:teuthology.orchestra.run.smithi164.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2022-11-27T16:12:28.389 INFO:teuthology.orchestra.run.smithi164.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 514 kB/s | 27 kB 00:00 2022-11-27T16:12:28.422 INFO:teuthology.orchestra.run.smithi164.stdout:(64/133): python3-kubernetes-11.0.0-6.el8.noarc 2.8 MB/s | 1.2 MB 00:00 2022-11-27T16:12:28.455 INFO:teuthology.orchestra.run.smithi164.stdout:(65/133): python3-waitress-1.2.1-2.el8.1.noarch 2.6 MB/s | 241 kB 00:00 2022-11-27T16:12:28.481 INFO:teuthology.orchestra.run.smithi164.stdout:(66/133): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2022-11-27T16:12:28.514 INFO:teuthology.orchestra.run.smithi164.stdout:(67/133): python3-webob-1.8.5-1.el8.1.noarch.rp 2.0 MB/s | 251 kB 00:00 2022-11-27T16:12:28.540 INFO:teuthology.orchestra.run.smithi164.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 392 kB/s | 23 kB 00:00 2022-11-27T16:12:28.565 INFO:teuthology.orchestra.run.smithi164.stdout:(69/133): python3-webtest-2.0.33-1.el8.noarch.r 784 kB/s | 85 kB 00:00 2022-11-27T16:12:28.590 INFO:teuthology.orchestra.run.smithi164.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2022-11-27T16:12:28.624 INFO:teuthology.orchestra.run.smithi167.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 3.5 MB/s | 24 MB 00:06 2022-11-27T16:12:28.649 INFO:teuthology.orchestra.run.smithi164.stdout:(71/133): postgresql-contrib-10.21-2.module+el8 9.5 MB/s | 808 kB 00:00 2022-11-27T16:12:28.670 INFO:teuthology.orchestra.run.smithi167.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 536 kB/s | 24 kB 00:00 2022-11-27T16:12:28.707 INFO:teuthology.orchestra.run.smithi164.stdout:(72/133): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2022-11-27T16:12:28.774 INFO:teuthology.orchestra.run.smithi164.stdout:(73/133): postgresql-10.21-2.module+el8.6.0+153 12 MB/s | 1.5 MB 00:00 2022-11-27T16:12:28.816 INFO:teuthology.orchestra.run.smithi164.stdout:(74/133): thrift-0.13.0-2.el8.x86_64.rpm 5.8 MB/s | 1.7 MB 00:00 2022-11-27T16:12:28.891 INFO:teuthology.orchestra.run.smithi164.stdout:(75/133): python3-numpy-1.14.3-10.el8.x86_64.rp 20 MB/s | 3.7 MB 00:00 2022-11-27T16:12:28.925 INFO:teuthology.orchestra.run.smithi164.stdout:(76/133): python3-werkzeug-0.12.2-4.el8.noarch. 4.1 MB/s | 457 kB 00:00 2022-11-27T16:12:28.959 INFO:teuthology.orchestra.run.smithi164.stdout:(77/133): python3-markupsafe-0.23-19.el8.x86_64 581 kB/s | 39 kB 00:00 2022-11-27T16:12:28.984 INFO:teuthology.orchestra.run.smithi164.stdout:(78/133): python3-prettytable-0.7.2-14.el8.noar 756 kB/s | 44 kB 00:00 2022-11-27T16:12:29.093 INFO:teuthology.orchestra.run.smithi164.stdout:(79/133): postgresql-server-10.21-2.module+el8. 16 MB/s | 5.1 MB 00:00 2022-11-27T16:12:29.120 INFO:teuthology.orchestra.run.smithi167.stdout:(10/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 5.5 MB/s | 18 MB 00:03 2022-11-27T16:12:29.126 INFO:teuthology.orchestra.run.smithi164.stdout:(80/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2022-11-27T16:12:29.160 INFO:teuthology.orchestra.run.smithi164.stdout:(81/133): python3-jinja2-2.10.1-3.el8.noarch.rp 3.0 MB/s | 538 kB 00:00 2022-11-27T16:12:29.202 INFO:teuthology.orchestra.run.smithi164.stdout:(82/133): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2022-11-27T16:12:29.219 INFO:teuthology.orchestra.run.smithi164.stdout:(83/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2022-11-27T16:12:29.222 INFO:teuthology.orchestra.run.smithi167.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 2.5 MB/s | 257 kB 00:00 2022-11-27T16:12:29.260 INFO:teuthology.orchestra.run.smithi164.stdout:(84/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.7 MB/s | 157 kB 00:00 2022-11-27T16:12:29.271 INFO:teuthology.orchestra.run.smithi167.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 638 kB/s | 30 kB 00:00 2022-11-27T16:12:29.303 INFO:teuthology.orchestra.run.smithi164.stdout:(85/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2022-11-27T16:12:29.378 INFO:teuthology.orchestra.run.smithi164.stdout:(86/133): openblas-threads-0.3.15-3.el8.x86_64. 23 MB/s | 4.9 MB 00:00 2022-11-27T16:12:29.403 INFO:teuthology.orchestra.run.smithi164.stdout:(87/133): socat-1.7.4.1-1.el8.x86_64.rpm 3.2 MB/s | 323 kB 00:00 2022-11-27T16:12:29.405 INFO:teuthology.orchestra.run.smithi167.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 5.6 MB/s | 767 kB 00:00 2022-11-27T16:12:29.456 INFO:teuthology.orchestra.run.smithi167.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 2.7 MB/s | 137 kB 00:00 2022-11-27T16:12:29.479 INFO:teuthology.orchestra.run.smithi164.stdout:(88/133): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2022-11-27T16:12:29.504 INFO:teuthology.orchestra.run.smithi164.stdout:(89/133): flex-2.6.1-9.el8.x86_64.rpm 2.5 MB/s | 320 kB 00:00 2022-11-27T16:12:29.537 INFO:teuthology.orchestra.run.smithi164.stdout:(90/133): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2022-11-27T16:12:29.539 INFO:teuthology.orchestra.run.smithi167.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.6 MB/s | 139 kB 00:00 2022-11-27T16:12:29.588 INFO:teuthology.orchestra.run.smithi164.stdout:(91/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-11-27T16:12:29.622 INFO:teuthology.orchestra.run.smithi167.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 5.6 MB/s | 474 kB 00:00 2022-11-27T16:12:29.655 INFO:teuthology.orchestra.run.smithi164.stdout:(92/133): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2022-11-27T16:12:29.688 INFO:teuthology.orchestra.run.smithi164.stdout:(93/133): python3-pytz-2017.2-9.el8.noarch.rpm 541 kB/s | 54 kB 00:00 2022-11-27T16:12:29.714 INFO:teuthology.orchestra.run.smithi164.stdout:(94/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.2 MB/s | 126 kB 00:00 2022-11-27T16:12:29.747 INFO:teuthology.orchestra.run.smithi164.stdout:(95/133): python3-jsonpointer-1.10-11.el8.noarc 342 kB/s | 20 kB 00:00 2022-11-27T16:12:29.797 INFO:teuthology.orchestra.run.smithi164.stdout:(96/133): libpq-13.5-1.el8.x86_64.rpm 2.3 MB/s | 198 kB 00:00 2022-11-27T16:12:29.839 INFO:teuthology.orchestra.run.smithi164.stdout:(97/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.8 MB/s | 259 kB 00:00 2022-11-27T16:12:29.865 INFO:teuthology.orchestra.run.smithi164.stdout:(98/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.8 MB/s | 187 kB 00:00 2022-11-27T16:12:29.899 INFO:teuthology.orchestra.run.smithi164.stdout:(99/133): python3-jsonpatch-1.21-2.el8.noarch.r 452 kB/s | 27 kB 00:00 2022-11-27T16:12:29.923 INFO:teuthology.orchestra.run.smithi164.stdout:(100/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-11-27T16:12:29.965 INFO:teuthology.orchestra.run.smithi164.stdout:(101/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.6 MB/s | 110 kB 00:00 2022-11-27T16:12:29.999 INFO:teuthology.orchestra.run.smithi164.stdout:(102/133): pcre2-devel-10.32-2.el8.x86_64.rpm 7.9 MB/s | 605 kB 00:00 2022-11-27T16:12:30.032 INFO:teuthology.orchestra.run.smithi164.stdout:(103/133): fuse-2.9.7-15.el8.x86_64.rpm 1.2 MB/s | 83 kB 00:00 2022-11-27T16:12:30.058 INFO:teuthology.orchestra.run.smithi164.stdout:(104/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2022-11-27T16:12:30.225 INFO:teuthology.orchestra.run.smithi164.stdout:(105/133): python3-scipy-1.0.0-21.module+el8.5. 19 MB/s | 14 MB 00:00 2022-11-27T16:12:30.250 INFO:teuthology.orchestra.run.smithi164.stdout:(106/133): libselinux-devel-2.9-5.el8.x86_64.rp 1.0 MB/s | 200 kB 00:00 2022-11-27T16:12:30.276 INFO:teuthology.orchestra.run.smithi164.stdout:(107/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 909 kB/s | 220 kB 00:00 2022-11-27T16:12:30.300 INFO:teuthology.orchestra.run.smithi164.stdout:(108/133): libverto-devel-0.3.0-5.el8.x86_64.rp 238 kB/s | 18 kB 00:00 2022-11-27T16:12:30.318 INFO:teuthology.orchestra.run.smithi164.stdout:(109/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 708 kB/s | 47 kB 00:00 2022-11-27T16:12:30.351 INFO:teuthology.orchestra.run.smithi164.stdout:(110/133): python3-cryptography-3.2.1-5.el8.x86 7.3 MB/s | 559 kB 00:00 2022-11-27T16:12:30.385 INFO:teuthology.orchestra.run.smithi164.stdout:(111/133): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2022-11-27T16:12:30.397 INFO:teuthology.orchestra.run.smithi167.stdout:(17/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 3.5 MB/s | 17 MB 00:04 2022-11-27T16:12:30.411 INFO:teuthology.orchestra.run.smithi164.stdout:(112/133): python3-pycparser-2.14-14.el8.noarch 1.2 MB/s | 109 kB 00:00 2022-11-27T16:12:30.444 INFO:teuthology.orchestra.run.smithi164.stdout:(113/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 6.1 MB/s | 560 kB 00:00 2022-11-27T16:12:30.469 INFO:teuthology.orchestra.run.smithi164.stdout:(114/133): python3-jwt-1.6.1-2.el8.noarch.rpm 727 kB/s | 43 kB 00:00 2022-11-27T16:12:30.472 INFO:teuthology.orchestra.run.smithi167.stdout:(18/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 7.6 MB/s | 6.5 MB 00:00 2022-11-27T16:12:30.494 INFO:teuthology.orchestra.run.smithi164.stdout:(115/133): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.5 MB/s | 171 kB 00:00 2022-11-27T16:12:30.498 INFO:teuthology.orchestra.run.smithi167.stdout:(19/133): python3-ceph-argparse-18.0.0-594.g697 446 kB/s | 45 kB 00:00 2022-11-27T16:12:30.523 INFO:teuthology.orchestra.run.smithi167.stdout:(20/133): python3-ceph-common-18.0.0-594.g697b0 2.1 MB/s | 109 kB 00:00 2022-11-27T16:12:30.528 INFO:teuthology.orchestra.run.smithi164.stdout:(116/133): zlib-devel-1.2.11-18.el8_5.x86_64.rp 689 kB/s | 58 kB 00:00 2022-11-27T16:12:30.573 INFO:teuthology.orchestra.run.smithi167.stdout:(21/133): python3-cephfs-18.0.0-594.g697b0d99.e 2.5 MB/s | 190 kB 00:00 2022-11-27T16:12:30.586 INFO:teuthology.orchestra.run.smithi164.stdout:(117/133): elfutils-libelf-devel-0.186-1.el8.x8 514 kB/s | 60 kB 00:00 2022-11-27T16:12:30.620 INFO:teuthology.orchestra.run.smithi164.stdout:(118/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.9 MB/s | 238 kB 00:00 2022-11-27T16:12:30.624 INFO:teuthology.orchestra.run.smithi167.stdout:(22/133): python3-rados-18.0.0-594.g697b0d99.el 3.6 MB/s | 371 kB 00:00 2022-11-27T16:12:30.662 INFO:teuthology.orchestra.run.smithi164.stdout:(119/133): libaio-devel-0.3.112-1.el8.x86_64.rp 139 kB/s | 19 kB 00:00 2022-11-27T16:12:30.687 INFO:teuthology.orchestra.run.smithi164.stdout:(120/133): libuuid-devel-2.32.1-35.el8.x86_64.r 972 kB/s | 98 kB 00:00 2022-11-27T16:12:30.690 INFO:teuthology.orchestra.run.smithi167.stdout:(23/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.6 MB/s | 107 kB 00:00 2022-11-27T16:12:30.713 INFO:teuthology.orchestra.run.smithi164.stdout:(121/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.5 MB/s | 229 kB 00:00 2022-11-27T16:12:30.716 INFO:teuthology.orchestra.run.smithi167.stdout:(24/133): python3-rbd-18.0.0-594.g697b0d99.el8. 2.5 MB/s | 357 kB 00:00 2022-11-27T16:12:30.742 INFO:teuthology.orchestra.run.smithi167.stdout:(25/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.8 MB/s | 94 kB 00:00 2022-11-27T16:12:30.746 INFO:teuthology.orchestra.run.smithi164.stdout:(122/133): python3-requests-oauthlib-1.0.0-1.el 510 kB/s | 43 kB 00:00 2022-11-27T16:12:30.771 INFO:teuthology.orchestra.run.smithi164.stdout:(123/133): libcom_err-devel-1.45.6-4.el8.x86_64 662 kB/s | 39 kB 00:00 2022-11-27T16:12:30.797 INFO:teuthology.orchestra.run.smithi164.stdout:(124/133): xfsprogs-devel-5.0.0-10.el8.x86_64.r 669 kB/s | 72 kB 00:00 2022-11-27T16:12:30.817 INFO:teuthology.orchestra.run.smithi167.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 2.3 MB/s | 175 kB 00:00 2022-11-27T16:12:30.839 INFO:teuthology.orchestra.run.smithi164.stdout:(125/133): keyutils-libs-devel-1.5.10-9.el8.x86 720 kB/s | 48 kB 00:00 2022-11-27T16:12:30.872 INFO:teuthology.orchestra.run.smithi164.stdout:(126/133): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.5 MB/s | 644 kB 00:00 2022-11-27T16:12:30.884 INFO:teuthology.orchestra.run.smithi167.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 344 kB/s | 23 kB 00:00 2022-11-27T16:12:30.941 INFO:teuthology.orchestra.run.smithi164.stdout:(127/133): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2022-11-27T16:12:30.974 INFO:teuthology.orchestra.run.smithi164.stdout:(128/133): NetworkManager-initscripts-updown-1. 1.0 MB/s | 138 kB 00:00 2022-11-27T16:12:30.976 INFO:teuthology.orchestra.run.smithi167.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.3 MB/s | 118 kB 00:00 2022-11-27T16:12:31.023 INFO:teuthology.orchestra.run.smithi164.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 14 MB/s | 709 kB 00:00 2022-11-27T16:12:31.074 INFO:teuthology.orchestra.run.smithi164.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 30 MB/s | 1.5 MB 00:00 2022-11-27T16:12:31.115 INFO:teuthology.orchestra.run.smithi164.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-11-27T16:12:31.263 INFO:teuthology.orchestra.run.smithi167.stdout:(29/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 6.4 MB/s | 3.5 MB 00:00 2022-11-27T16:12:31.476 INFO:teuthology.orchestra.run.smithi167.stdout:(30/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 3.3 MB/s | 1.7 MB 00:00 2022-11-27T16:12:31.515 INFO:teuthology.orchestra.run.smithi164.stdout:(132/133): librados2-18.0.0-594.g697b0d99.el8.x 5.6 MB/s | 3.6 MB 00:00 2022-11-27T16:12:31.566 INFO:teuthology.orchestra.run.smithi164.stdout:(133/133): librbd1-18.0.0-594.g697b0d99.el8.x86 5.7 MB/s | 3.6 MB 00:00 2022-11-27T16:12:31.568 INFO:teuthology.orchestra.run.smithi164.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:12:31.569 INFO:teuthology.orchestra.run.smithi164.stdout:Total 20 MB/s | 219 MB 00:10 2022-11-27T16:12:31.585 INFO:teuthology.orchestra.run.smithi167.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.1 MB/s | 234 kB 00:00 2022-11-27T16:12:31.635 INFO:teuthology.orchestra.run.smithi167.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 941 kB/s | 47 kB 00:00 2022-11-27T16:12:31.694 INFO:teuthology.orchestra.run.smithi167.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 248 kB/s | 14 kB 00:00 2022-11-27T16:12:31.786 INFO:teuthology.orchestra.run.smithi167.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 2.8 MB/s | 263 kB 00:00 2022-11-27T16:12:31.870 INFO:teuthology.orchestra.run.smithi167.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 975 kB/s | 81 kB 00:00 2022-11-27T16:12:31.920 INFO:teuthology.orchestra.run.smithi167.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 311 kB/s | 15 kB 00:00 2022-11-27T16:12:32.131 INFO:teuthology.orchestra.run.smithi167.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.4 MB/s | 306 kB 00:00 2022-11-27T16:12:32.302 INFO:teuthology.orchestra.run.smithi167.stdout:(38/133): ceph-mgr-diskprediction-local-18.0.0- 7.1 MB/s | 7.4 MB 00:01 2022-11-27T16:12:32.451 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction check 2022-11-27T16:12:32.479 INFO:teuthology.orchestra.run.smithi167.stdout:(39/133): liboath-2.6.2-3.el8.x86_64.rpm 169 kB/s | 59 kB 00:00 2022-11-27T16:12:32.588 INFO:teuthology.orchestra.run.smithi167.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2022-11-27T16:12:32.612 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction check succeeded. 2022-11-27T16:12:32.613 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction test 2022-11-27T16:12:32.613 INFO:teuthology.orchestra.run.smithi167.stdout:(41/133): libunwind-1.3.1-3.el8.x86_64.rpm 242 kB/s | 75 kB 00:00 2022-11-27T16:12:32.638 INFO:teuthology.orchestra.run.smithi167.stdout:(42/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 873 kB/s | 44 kB 00:00 2022-11-27T16:12:32.713 INFO:teuthology.orchestra.run.smithi167.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 447 kB/s | 33 kB 00:00 2022-11-27T16:12:32.747 INFO:teuthology.orchestra.run.smithi167.stdout:(44/133): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2022-11-27T16:12:32.764 INFO:teuthology.orchestra.run.smithi167.stdout:(45/133): python3-certifi-2018.10.15-7.el8.noar 317 kB/s | 16 kB 00:00 2022-11-27T16:12:32.798 INFO:teuthology.orchestra.run.smithi167.stdout:(46/133): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2022-11-27T16:12:32.848 INFO:teuthology.orchestra.run.smithi167.stdout:(47/133): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2022-11-27T16:12:32.873 INFO:teuthology.orchestra.run.smithi167.stdout:(48/133): python3-google-auth-1.1.1-10.el8.noar 1.1 MB/s | 81 kB 00:00 2022-11-27T16:12:32.891 INFO:teuthology.orchestra.run.smithi167.stdout:(49/133): python3-jaraco-6.2-6.el8.noarch.rpm 265 kB/s | 11 kB 00:00 2022-11-27T16:12:32.916 INFO:teuthology.orchestra.run.smithi167.stdout:(50/133): python3-jaraco-functools-2.0-4.el8.no 419 kB/s | 18 kB 00:00 2022-11-27T16:12:33.008 INFO:teuthology.orchestra.run.smithi167.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 533 kB/s | 49 kB 00:00 2022-11-27T16:12:33.067 INFO:teuthology.orchestra.run.smithi167.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 998 kB/s | 59 kB 00:00 2022-11-27T16:12:33.100 INFO:teuthology.orchestra.run.smithi167.stdout:(53/133): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2022-11-27T16:12:33.125 INFO:teuthology.orchestra.run.smithi167.stdout:(54/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.0 MB/s | 60 kB 00:00 2022-11-27T16:12:33.159 INFO:teuthology.orchestra.run.smithi167.stdout:(55/133): python3-pecan-1.3.2-9.el8.noarch.rpm 4.7 MB/s | 283 kB 00:00 2022-11-27T16:12:33.184 INFO:teuthology.orchestra.run.smithi167.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 265 kB/s | 16 kB 00:00 2022-11-27T16:12:33.210 INFO:teuthology.orchestra.run.smithi167.stdout:(57/133): python3-repoze-lru-0.7-6.el8.noarch.r 673 kB/s | 34 kB 00:00 2022-11-27T16:12:33.235 INFO:teuthology.orchestra.run.smithi167.stdout:(58/133): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2022-11-27T16:12:33.260 INFO:teuthology.orchestra.run.smithi167.stdout:(59/133): python3-rsa-4.8-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2022-11-27T16:12:33.311 INFO:teuthology.orchestra.run.smithi167.stdout:(60/133): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2022-11-27T16:12:33.336 INFO:teuthology.orchestra.run.smithi167.stdout:(61/133): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2022-11-27T16:12:33.362 INFO:teuthology.orchestra.run.smithi167.stdout:(62/133): python3-tempora-1.14.1-5.el8.noarch.r 582 kB/s | 29 kB 00:00 2022-11-27T16:12:33.386 INFO:teuthology.orchestra.run.smithi167.stdout:(63/133): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2022-11-27T16:12:33.412 INFO:teuthology.orchestra.run.smithi167.stdout:(64/133): python3-waitress-1.2.1-2.el8.1.noarch 4.7 MB/s | 241 kB 00:00 2022-11-27T16:12:33.437 INFO:teuthology.orchestra.run.smithi167.stdout:(65/133): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2022-11-27T16:12:33.462 INFO:teuthology.orchestra.run.smithi167.stdout:(66/133): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2022-11-27T16:12:33.488 INFO:teuthology.orchestra.run.smithi167.stdout:(67/133): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2022-11-27T16:12:33.513 INFO:teuthology.orchestra.run.smithi167.stdout:(68/133): python3-zc-lockfile-2.0-2.el8.noarch. 456 kB/s | 23 kB 00:00 2022-11-27T16:12:33.538 INFO:teuthology.orchestra.run.smithi167.stdout:(69/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2022-11-27T16:12:33.615 INFO:teuthology.orchestra.run.smithi167.stdout:(70/133): pcre2-devel-10.32-2.el8.x86_64.rpm 7.9 MB/s | 605 kB 00:00 2022-11-27T16:12:33.656 INFO:teuthology.orchestra.run.smithi167.stdout:(71/133): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2022-11-27T16:12:33.698 INFO:teuthology.orchestra.run.smithi167.stdout:(72/133): libsepol-devel-2.9-3.el8.x86_64.rpm 2.1 MB/s | 87 kB 00:00 2022-11-27T16:12:33.740 INFO:teuthology.orchestra.run.smithi167.stdout:(73/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2022-11-27T16:12:33.782 INFO:teuthology.orchestra.run.smithi167.stdout:(74/133): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2022-11-27T16:12:33.782 INFO:teuthology.orchestra.run.smithi164.stdout:Transaction test succeeded. 2022-11-27T16:12:33.787 INFO:teuthology.orchestra.run.smithi164.stdout:Running transaction 2022-11-27T16:12:33.815 INFO:teuthology.orchestra.run.smithi167.stdout:(75/133): libselinux-devel-2.9-5.el8.x86_64.rpm 2.6 MB/s | 200 kB 00:00 2022-11-27T16:12:33.883 INFO:teuthology.orchestra.run.smithi167.stdout:(76/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 179 kB/s | 18 kB 00:00 2022-11-27T16:12:33.907 INFO:teuthology.orchestra.run.smithi167.stdout:(77/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 516 kB/s | 47 kB 00:00 2022-11-27T16:12:33.950 INFO:teuthology.orchestra.run.smithi167.stdout:(78/133): python3-cryptography-3.2.1-5.el8.x86_ 8.1 MB/s | 559 kB 00:00 2022-11-27T16:12:33.975 INFO:teuthology.orchestra.run.smithi167.stdout:(79/133): python3-oauthlib-2.1.0-1.el8.noarch.r 2.3 MB/s | 155 kB 00:00 2022-11-27T16:12:34.017 INFO:teuthology.orchestra.run.smithi167.stdout:(80/133): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2022-11-27T16:12:34.076 INFO:teuthology.orchestra.run.smithi167.stdout:(81/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.5 MB/s | 560 kB 00:00 2022-11-27T16:12:34.109 INFO:teuthology.orchestra.run.smithi167.stdout:(82/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.8 MB/s | 171 kB 00:00 2022-11-27T16:12:34.135 INFO:teuthology.orchestra.run.smithi167.stdout:(83/133): python3-jwt-1.6.1-2.el8.noarch.rpm 744 kB/s | 43 kB 00:00 2022-11-27T16:12:34.193 INFO:teuthology.orchestra.run.smithi167.stdout:(84/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 690 kB/s | 58 kB 00:00 2022-11-27T16:12:34.218 INFO:teuthology.orchestra.run.smithi167.stdout:(85/133): elfutils-libelf-devel-0.186-1.el8.x86 716 kB/s | 60 kB 00:00 2022-11-27T16:12:34.252 INFO:teuthology.orchestra.run.smithi167.stdout:(86/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2022-11-27T16:12:34.278 INFO:teuthology.orchestra.run.smithi167.stdout:(87/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2022-11-27T16:12:34.311 INFO:teuthology.orchestra.run.smithi167.stdout:(88/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.6 MB/s | 98 kB 00:00 2022-11-27T16:12:34.336 INFO:teuthology.orchestra.run.smithi167.stdout:(89/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.8 MB/s | 229 kB 00:00 2022-11-27T16:12:34.370 INFO:teuthology.orchestra.run.smithi167.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 728 kB/s | 43 kB 00:00 2022-11-27T16:12:34.395 INFO:teuthology.orchestra.run.smithi167.stdout:(91/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.2 MB/s | 72 kB 00:00 2022-11-27T16:12:34.429 INFO:teuthology.orchestra.run.smithi167.stdout:(92/133): libcom_err-devel-1.45.6-4.el8.x86_64. 659 kB/s | 39 kB 00:00 2022-11-27T16:12:34.488 INFO:teuthology.orchestra.run.smithi167.stdout:(93/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 827 kB/s | 48 kB 00:00 2022-11-27T16:12:34.538 INFO:teuthology.orchestra.run.smithi167.stdout:(94/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2022-11-27T16:12:34.580 INFO:teuthology.orchestra.run.smithi167.stdout:(95/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 6.8 MB/s | 644 kB 00:00 2022-11-27T16:12:34.605 INFO:teuthology.orchestra.run.smithi167.stdout:(96/133): NetworkManager-initscripts-updown-1.3 2.0 MB/s | 138 kB 00:00 2022-11-27T16:12:34.647 INFO:teuthology.orchestra.run.smithi167.stdout:(97/133): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2022-11-27T16:12:34.689 INFO:teuthology.orchestra.run.smithi167.stdout:(98/133): bison-3.0.4-10.el8.x86_64.rpm 8.2 MB/s | 688 kB 00:00 2022-11-27T16:12:34.781 INFO:teuthology.orchestra.run.smithi167.stdout:(99/133): postgresql-10.21-2.module+el8.6.0+153 11 MB/s | 1.5 MB 00:00 2022-11-27T16:12:34.857 INFO:teuthology.orchestra.run.smithi167.stdout:(100/133): python3-numpy-1.14.3-10.el8.x86_64.r 23 MB/s | 3.7 MB 00:00 2022-11-27T16:12:34.915 INFO:teuthology.orchestra.run.smithi167.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2022-11-27T16:12:34.982 INFO:teuthology.orchestra.run.smithi167.stdout:(102/133): postgresql-server-10.21-2.module+el8 25 MB/s | 5.1 MB 00:00 2022-11-27T16:12:34.999 INFO:teuthology.orchestra.run.smithi167.stdout:(103/133): python3-markupsafe-0.23-19.el8.x86_6 466 kB/s | 39 kB 00:00 2022-11-27T16:12:35.516 INFO:teuthology.orchestra.run.smithi167.stdout:(104/133): ceph-test-18.0.0-594.g697b0d99.el8.x 8.0 MB/s | 55 MB 00:06 2022-11-27T16:12:35.559 INFO:teuthology.orchestra.run.smithi167.stdout:(105/133): python3-prettytable-0.7.2-14.el8.noa 76 kB/s | 44 kB 00:00 2022-11-27T16:12:35.592 INFO:teuthology.orchestra.run.smithi167.stdout:(106/133): librdkafka-0.11.4-3.el8.x86_64.rpm 597 kB/s | 354 kB 00:00 2022-11-27T16:12:35.626 INFO:teuthology.orchestra.run.smithi167.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 4.9 MB/s | 538 kB 00:00 2022-11-27T16:12:35.668 INFO:teuthology.orchestra.run.smithi167.stdout:(108/133): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2022-11-27T16:12:35.693 INFO:teuthology.orchestra.run.smithi167.stdout:(109/133): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2022-11-27T16:12:35.735 INFO:teuthology.orchestra.run.smithi167.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2022-11-27T16:12:35.798 INFO:teuthology.orchestra.run.smithi167.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 1.6 MB/s | 103 kB 00:00 2022-11-27T16:12:35.887 INFO:teuthology.orchestra.run.smithi167.stdout:(112/133): openblas-threads-0.3.15-3.el8.x86_64 19 MB/s | 4.9 MB 00:00 2022-11-27T16:12:35.919 INFO:teuthology.orchestra.run.smithi167.stdout:(113/133): socat-1.7.4.1-1.el8.x86_64.rpm 2.6 MB/s | 323 kB 00:00 2022-11-27T16:12:35.952 INFO:teuthology.orchestra.run.smithi167.stdout:(114/133): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2022-11-27T16:12:36.019 INFO:teuthology.orchestra.run.smithi167.stdout:(115/133): python3-babel-2.5.1-7.el8.noarch.rpm 15 MB/s | 4.8 MB 00:00 2022-11-27T16:12:36.062 INFO:teuthology.orchestra.run.smithi167.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 1.8 MB/s | 202 kB 00:00 2022-11-27T16:12:36.120 INFO:teuthology.orchestra.run.smithi167.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2022-11-27T16:12:36.187 INFO:teuthology.orchestra.run.smithi167.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2022-11-27T16:12:36.220 INFO:teuthology.orchestra.run.smithi167.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 542 kB/s | 54 kB 00:00 2022-11-27T16:12:36.246 INFO:teuthology.orchestra.run.smithi167.stdout:(120/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.2 MB/s | 126 kB 00:00 2022-11-27T16:12:36.279 INFO:teuthology.orchestra.run.smithi167.stdout:(121/133): python3-jsonpointer-1.10-11.el8.noar 341 kB/s | 20 kB 00:00 2022-11-27T16:12:36.305 INFO:teuthology.orchestra.run.smithi167.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 3.4 MB/s | 198 kB 00:00 2022-11-27T16:12:36.346 INFO:teuthology.orchestra.run.smithi167.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2022-11-27T16:12:36.397 INFO:teuthology.orchestra.run.smithi167.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2022-11-27T16:12:36.431 INFO:teuthology.orchestra.run.smithi167.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 317 kB/s | 27 kB 00:00 2022-11-27T16:12:36.456 INFO:teuthology.orchestra.run.smithi167.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-11-27T16:12:36.498 INFO:teuthology.orchestra.run.smithi167.stdout:(127/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.6 MB/s | 110 kB 00:00 2022-11-27T16:12:36.632 INFO:teuthology.orchestra.run.smithi167.stdout:(128/133): python3-scipy-1.0.0-21.module+el8.5. 22 MB/s | 14 MB 00:00 2022-11-27T16:12:36.699 INFO:teuthology.orchestra.run.smithi167.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 10 MB/s | 709 kB 00:00 2022-11-27T16:12:36.758 INFO:teuthology.orchestra.run.smithi167.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-11-27T16:12:36.800 INFO:teuthology.orchestra.run.smithi167.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 19 MB/s | 798 kB 00:00 2022-11-27T16:12:37.041 INFO:teuthology.orchestra.run.smithi167.stdout:(132/133): librbd1-18.0.0-594.g697b0d99.el8.x86 6.6 MB/s | 3.6 MB 00:00 2022-11-27T16:12:37.216 INFO:teuthology.orchestra.run.smithi167.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 4.8 MB/s | 3.6 MB 00:00 2022-11-27T16:12:37.220 INFO:teuthology.orchestra.run.smithi167.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:12:37.220 INFO:teuthology.orchestra.run.smithi167.stdout:Total 14 MB/s | 219 MB 00:15 2022-11-27T16:12:38.107 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction check 2022-11-27T16:12:38.267 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction check succeeded. 2022-11-27T16:12:38.267 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction test 2022-11-27T16:12:38.916 INFO:teuthology.orchestra.run.smithi164.stdout: Preparing : 1/1 2022-11-27T16:12:39.142 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:12:39.356 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:12:39.438 INFO:teuthology.orchestra.run.smithi167.stdout:Transaction test succeeded. 2022-11-27T16:12:39.442 INFO:teuthology.orchestra.run.smithi167.stdout:Running transaction 2022-11-27T16:12:39.551 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T16:12:39.717 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T16:12:39.853 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T16:12:39.917 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T16:12:40.162 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 5/138 2022-11-27T16:12:40.312 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:12:40.705 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:12:40.978 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:12:41.370 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:12:41.552 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:12:41.683 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:12:41.870 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:12:41.996 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:12:42.175 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:12:42.431 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T16:12:42.581 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T16:12:42.947 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T16:12:43.190 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T16:12:43.353 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:12:43.485 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:12:43.669 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:12:43.883 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:12:44.042 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 17/138 2022-11-27T16:12:44.144 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T16:12:44.282 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T16:12:44.386 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T16:12:44.945 INFO:teuthology.orchestra.run.smithi167.stdout: Preparing : 1/1 2022-11-27T16:12:45.015 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T16:12:45.169 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T16:12:45.181 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:12:45.430 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:12:45.630 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T16:12:45.761 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T16:12:45.765 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T16:12:45.927 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T16:12:45.980 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-11-27T16:12:46.166 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T16:12:46.265 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T16:12:46.465 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:12:46.629 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T16:12:46.849 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:12:46.941 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 22/138 2022-11-27T16:12:47.122 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 23/138 2022-11-27T16:12:47.132 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:12:47.419 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/138 2022-11-27T16:12:47.524 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:12:47.698 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/138 2022-11-27T16:12:47.740 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:12:47.850 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:12:47.985 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T16:12:48.073 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:12:48.173 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:12:48.183 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T16:12:48.312 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T16:12:48.388 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:12:48.529 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T16:12:48.984 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T16:12:49.162 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T16:12:49.427 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/138 2022-11-27T16:12:49.555 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T16:12:49.836 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T16:12:49.865 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/138 2022-11-27T16:12:49.998 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:12:50.137 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:12:50.164 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/138 2022-11-27T16:12:50.306 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:12:50.395 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T16:12:50.410 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:12:50.560 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-11-27T16:12:50.822 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-11-27T16:12:51.052 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-11-27T16:12:51.212 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-11-27T16:12:51.365 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T16:12:51.502 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T16:12:51.657 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T16:12:52.124 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T16:12:52.262 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T16:12:52.486 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/138 2022-11-27T16:12:52.490 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T16:12:52.729 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T16:12:52.927 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T16:12:53.071 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T16:12:53.199 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T16:12:53.265 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T16:12:53.429 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T16:12:53.638 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T16:12:53.805 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 40/138 2022-11-27T16:12:53.975 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T16:12:53.976 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 41/138 2022-11-27T16:12:54.192 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/138 2022-11-27T16:12:54.327 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-11-27T16:12:54.398 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 43/138 2022-11-27T16:12:54.606 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T16:12:54.649 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 44/138 2022-11-27T16:12:54.812 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T16:12:54.941 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T16:12:55.044 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 45/138 2022-11-27T16:12:55.234 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T16:12:55.345 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 46/138 2022-11-27T16:12:55.448 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T16:12:55.535 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 47/138 2022-11-27T16:12:55.811 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 48/138 2022-11-27T16:12:56.088 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 49/138 2022-11-27T16:12:56.390 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 50/138 2022-11-27T16:12:56.583 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 51/138 2022-11-27T16:12:56.802 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 52/138 2022-11-27T16:12:57.028 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 53/138 2022-11-27T16:12:57.172 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T16:12:57.204 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 54/138 2022-11-27T16:12:57.508 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/138 2022-11-27T16:12:57.581 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-11-27T16:12:57.696 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/138 2022-11-27T16:12:57.970 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/138 2022-11-27T16:12:58.196 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/138 2022-11-27T16:12:58.410 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 59/138 2022-11-27T16:12:58.538 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-11-27T16:12:58.610 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : fuse-2.9.7-15.el8.x86_64 60/138 2022-11-27T16:12:58.775 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T16:12:58.858 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T16:12:58.884 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-11-27T16:12:59.063 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T16:12:59.140 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T16:12:59.173 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-11-27T16:12:59.332 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : jq-1.6-3.el8.x86_64 63/138 2022-11-27T16:12:59.408 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T16:12:59.516 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 64/138 2022-11-27T16:12:59.590 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T16:12:59.808 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 65/138 2022-11-27T16:12:59.826 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T16:13:00.059 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T16:13:00.067 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : automake-1.16.1-7.el8.noarch 66/138 2022-11-27T16:13:00.219 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 67/138 2022-11-27T16:13:00.297 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T16:13:00.409 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T16:13:00.480 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T16:13:00.567 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T16:13:00.579 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:13:00.755 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T16:13:00.824 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T16:13:00.892 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T16:13:01.028 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-11-27T16:13:01.132 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T16:13:01.223 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-11-27T16:13:01.305 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T16:13:01.458 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-11-27T16:13:01.511 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T16:13:01.688 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-11-27T16:13:01.717 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T16:13:01.900 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-11-27T16:13:01.955 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T16:13:02.017 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-11-27T16:13:02.136 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T16:13:02.225 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-11-27T16:13:02.336 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T16:13:02.453 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-11-27T16:13:02.472 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T16:13:02.652 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-11-27T16:13:02.712 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T16:13:02.910 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-11-27T16:13:02.911 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T16:13:03.122 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T16:13:03.357 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-11-27T16:13:03.361 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T16:13:03.582 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T16:13:03.730 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-11-27T16:13:03.791 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T16:13:03.958 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-11-27T16:13:04.007 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T16:13:04.233 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-11-27T16:13:04.533 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-11-27T16:13:04.892 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-11-27T16:13:05.090 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T16:13:05.180 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-11-27T16:13:05.370 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-11-27T16:13:05.386 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T16:13:05.623 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-11-27T16:13:05.676 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T16:13:05.799 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-11-27T16:13:05.998 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T16:13:06.160 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-11-27T16:13:06.267 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T16:13:06.391 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-11-27T16:13:06.441 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T16:13:06.691 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-11-27T16:13:06.975 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-11-27T16:13:07.219 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T16:13:07.255 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-11-27T16:13:07.407 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:13:07.463 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-11-27T16:13:07.494 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:13:07.645 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T16:13:07.708 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T16:13:07.873 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:13:08.193 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T16:13:08.453 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T16:13:08.675 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T16:13:08.848 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T16:13:09.088 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T16:13:09.341 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T16:13:09.574 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T16:13:09.791 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T16:13:09.976 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T16:13:10.241 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T16:13:10.506 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T16:13:10.719 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T16:13:10.997 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T16:13:11.211 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T16:13:11.436 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T16:13:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:13:11.669 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T16:13:11.790 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:13:12.348 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:13:12.662 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:13:12.745 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:13:12.750 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T16:13:12.898 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:13:13.015 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T16:13:13.284 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T16:13:13.609 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T16:13:13.920 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T16:13:14.127 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T16:13:14.916 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T16:13:15.203 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:13:15.263 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:13:15.504 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T16:13:19.372 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:13:19.629 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:13:20.186 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:13:20.692 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:13:20.789 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:13:20.919 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:13:31.703 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:13:31.704 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /sys 2022-11-27T16:13:31.704 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /proc 2022-11-27T16:13:31.705 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /dev 2022-11-27T16:13:31.705 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /run 2022-11-27T16:13:31.705 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /mnt 2022-11-27T16:13:31.706 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /var/tmp 2022-11-27T16:13:31.706 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /home 2022-11-27T16:13:31.707 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /tmp 2022-11-27T16:13:31.707 INFO:teuthology.orchestra.run.smithi164.stdout:skipping the directory /dev 2022-11-27T16:13:31.708 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:13:32.079 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:13:33.560 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:13:33.767 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:13:33.984 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:13:34.154 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:13:34.535 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:13:34.712 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:13:34.729 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:13:36.376 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:13:36.544 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:13:36.668 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T16:13:36.890 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T16:13:37.053 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:13:37.261 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:13:37.592 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:13:38.553 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:13:38.735 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:13:38.826 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:13:39.192 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T16:13:39.405 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:13:39.440 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:13:39.619 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:13:39.630 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:13:39.929 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:13:39.929 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /sys 2022-11-27T16:13:39.929 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /proc 2022-11-27T16:13:39.929 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2022-11-27T16:13:39.930 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /run 2022-11-27T16:13:39.930 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /mnt 2022-11-27T16:13:39.930 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /var/tmp 2022-11-27T16:13:39.930 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /home 2022-11-27T16:13:39.931 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /tmp 2022-11-27T16:13:39.931 INFO:teuthology.orchestra.run.smithi167.stdout:skipping the directory /dev 2022-11-27T16:13:39.931 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:13:40.333 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:13:41.389 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T16:13:41.609 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:13:41.643 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:13:41.785 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:13:41.816 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:13:41.828 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:13:41.994 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:13:41.997 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:13:42.214 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:13:42.341 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:13:42.416 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:13:42.502 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:13:42.797 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:13:42.991 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:13:43.009 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:13:44.680 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:13:44.875 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:13:44.988 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T16:13:45.186 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T16:13:45.390 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:13:45.596 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:13:45.980 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:13:46.943 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:13:47.148 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:13:47.238 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:13:47.591 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:13:47.623 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T16:13:47.832 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:13:47.867 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:13:47.903 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T16:13:48.076 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T16:13:48.081 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:13:48.093 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:13:48.266 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T16:13:48.607 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/138 2022-11-27T16:13:49.406 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 119/138 2022-11-27T16:13:49.652 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/138 2022-11-27T16:13:49.797 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/138 2022-11-27T16:13:49.799 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T16:13:49.987 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:13:50.005 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T16:13:50.021 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:13:50.127 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T16:13:50.165 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:13:50.207 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:13:50.382 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:13:50.728 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:13:50.874 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:13:52.220 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:13:52.526 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:13:52.543 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:13:52.730 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T16:13:52.966 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T16:13:53.224 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T16:13:53.609 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T16:13:53.933 INFO:teuthology.orchestra.run.smithi164.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T16:13:54.084 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T16:13:54.092 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T16:13:54.252 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/138 2022-11-27T16:13:54.525 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T16:13:54.601 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T16:13:54.824 INFO:teuthology.orchestra.run.smithi164.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T16:13:54.840 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T16:13:55.025 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:13:55.026 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:13:55.236 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:13:55.237 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:13:55.486 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:13:55.486 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:13:55.720 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:13:55.915 INFO:teuthology.orchestra.run.smithi164.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:13:55.973 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:13:56.311 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T16:13:56.511 INFO:teuthology.orchestra.run.smithi164.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:13:56.511 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T16:13:56.512 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T16:13:56.512 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T16:13:56.513 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T16:13:56.513 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T16:13:56.513 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T16:13:56.513 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:13:56.514 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:13:56.514 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:13:56.514 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:13:56.514 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T16:13:56.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T16:13:56.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T16:13:56.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T16:13:56.515 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:13:56.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T16:13:56.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T16:13:56.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T16:13:56.516 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T16:13:56.517 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T16:13:56.517 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T16:13:56.517 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T16:13:56.517 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T16:13:56.518 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T16:13:56.518 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T16:13:56.518 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T16:13:56.518 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T16:13:56.519 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T16:13:56.519 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T16:13:56.519 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T16:13:56.519 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T16:13:56.520 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T16:13:56.520 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T16:13:56.520 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T16:13:56.521 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T16:13:56.521 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T16:13:56.521 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T16:13:56.522 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T16:13:56.522 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T16:13:56.523 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T16:13:56.523 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T16:13:56.523 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T16:13:56.523 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T16:13:56.524 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T16:13:56.524 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T16:13:56.524 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T16:13:56.524 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T16:13:56.525 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T16:13:56.525 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T16:13:56.526 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T16:13:56.526 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T16:13:56.527 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T16:13:56.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T16:13:56.528 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T16:13:56.529 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T16:13:56.529 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T16:13:56.530 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T16:13:56.530 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T16:13:56.530 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T16:13:56.531 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T16:13:56.532 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T16:13:56.532 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T16:13:56.533 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T16:13:56.534 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T16:13:56.534 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T16:13:56.534 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T16:13:56.535 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T16:13:56.535 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T16:13:56.536 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:13:56.536 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T16:13:56.536 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T16:13:56.536 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 72/138 2022-11-27T16:13:56.537 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : bison-3.0.4-10.el8.x86_64 73/138 2022-11-27T16:13:56.537 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 74/138 2022-11-27T16:13:56.537 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 75/138 2022-11-27T16:13:56.537 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 76/138 2022-11-27T16:13:56.537 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/138 2022-11-27T16:13:56.538 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/138 2022-11-27T16:13:56.538 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 79/138 2022-11-27T16:13:56.538 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 80/138 2022-11-27T16:13:56.538 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 81/138 2022-11-27T16:13:56.539 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/138 2022-11-27T16:13:56.539 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 83/138 2022-11-27T16:13:56.539 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 84/138 2022-11-27T16:13:56.540 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 85/138 2022-11-27T16:13:56.540 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/138 2022-11-27T16:13:56.540 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/138 2022-11-27T16:13:56.540 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/138 2022-11-27T16:13:56.540 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : flex-2.6.1-9.el8.x86_64 89/138 2022-11-27T16:13:56.541 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 90/138 2022-11-27T16:13:56.541 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : jq-1.6-3.el8.x86_64 91/138 2022-11-27T16:13:56.541 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 92/138 2022-11-27T16:13:56.541 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : automake-1.16.1-7.el8.noarch 93/138 2022-11-27T16:13:56.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 94/138 2022-11-27T16:13:56.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 95/138 2022-11-27T16:13:56.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 96/138 2022-11-27T16:13:56.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/138 2022-11-27T16:13:56.542 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 98/138 2022-11-27T16:13:56.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 99/138 2022-11-27T16:13:56.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 100/138 2022-11-27T16:13:56.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 101/138 2022-11-27T16:13:56.543 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/138 2022-11-27T16:13:56.544 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 103/138 2022-11-27T16:13:56.547 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 104/138 2022-11-27T16:13:56.548 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 105/138 2022-11-27T16:13:56.548 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/138 2022-11-27T16:13:56.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 107/138 2022-11-27T16:13:56.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 108/138 2022-11-27T16:13:56.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 109/138 2022-11-27T16:13:56.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 110/138 2022-11-27T16:13:56.549 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 111/138 2022-11-27T16:13:56.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 112/138 2022-11-27T16:13:56.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/138 2022-11-27T16:13:56.550 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 114/138 2022-11-27T16:13:56.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 115/138 2022-11-27T16:13:56.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/138 2022-11-27T16:13:56.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 117/138 2022-11-27T16:13:56.551 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/138 2022-11-27T16:13:56.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 119/138 2022-11-27T16:13:56.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 120/138 2022-11-27T16:13:56.552 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 121/138 2022-11-27T16:13:56.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 122/138 2022-11-27T16:13:56.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 123/138 2022-11-27T16:13:56.553 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 124/138 2022-11-27T16:13:56.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 125/138 2022-11-27T16:13:56.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 126/138 2022-11-27T16:13:56.554 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/138 2022-11-27T16:13:56.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/138 2022-11-27T16:13:56.555 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T16:13:56.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T16:13:56.556 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T16:13:56.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T16:13:56.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T16:13:56.557 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:13:56.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T16:13:56.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T16:13:56.558 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T16:13:57.144 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 117/138 2022-11-27T16:13:57.432 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-11-27T16:13:57.640 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-11-27T16:13:57.797 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T16:13:57.986 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T16:13:58.260 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-11-27T16:13:58.459 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T16:13:58.570 INFO:teuthology.orchestra.run.smithi164.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:13:58.571 INFO:teuthology.orchestra.run.smithi164.stdout:Installed products updated. 2022-11-27T16:13:58.631 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T16:13:58.814 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:13:58.814 INFO:teuthology.orchestra.run.smithi164.stdout:Upgraded: 2022-11-27T16:13:58.815 INFO:teuthology.orchestra.run.smithi164.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T16:13:58.815 INFO:teuthology.orchestra.run.smithi164.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.815 INFO:teuthology.orchestra.run.smithi164.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.815 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:13:58.816 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:13:58.816 INFO:teuthology.orchestra.run.smithi164.stdout:Installed: 2022-11-27T16:13:58.816 INFO:teuthology.orchestra.run.smithi164.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T16:13:58.816 INFO:teuthology.orchestra.run.smithi164.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:13:58.817 INFO:teuthology.orchestra.run.smithi164.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:13:58.817 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.817 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.817 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.817 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.818 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.818 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.818 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.818 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.819 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.819 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.819 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.819 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.819 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.820 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.820 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.820 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.820 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.821 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.821 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.822 INFO:teuthology.orchestra.run.smithi164.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.822 INFO:teuthology.orchestra.run.smithi164.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.822 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.822 INFO:teuthology.orchestra.run.smithi164.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:13:58.822 INFO:teuthology.orchestra.run.smithi164.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T16:13:58.823 INFO:teuthology.orchestra.run.smithi164.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:13:58.823 INFO:teuthology.orchestra.run.smithi164.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T16:13:58.823 INFO:teuthology.orchestra.run.smithi164.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:13:58.823 INFO:teuthology.orchestra.run.smithi164.stdout: jq-1.6-3.el8.x86_64 2022-11-27T16:13:58.824 INFO:teuthology.orchestra.run.smithi164.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:13:58.824 INFO:teuthology.orchestra.run.smithi164.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T16:13:58.824 INFO:teuthology.orchestra.run.smithi164.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:13:58.824 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.825 INFO:teuthology.orchestra.run.smithi164.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.825 INFO:teuthology.orchestra.run.smithi164.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.825 INFO:teuthology.orchestra.run.smithi164.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T16:13:58.825 INFO:teuthology.orchestra.run.smithi164.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:13:58.825 INFO:teuthology.orchestra.run.smithi164.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:13:58.826 INFO:teuthology.orchestra.run.smithi164.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:13:58.826 INFO:teuthology.orchestra.run.smithi164.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:13:58.826 INFO:teuthology.orchestra.run.smithi164.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:13:58.826 INFO:teuthology.orchestra.run.smithi164.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.827 INFO:teuthology.orchestra.run.smithi164.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.827 INFO:teuthology.orchestra.run.smithi164.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:13:58.827 INFO:teuthology.orchestra.run.smithi164.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.827 INFO:teuthology.orchestra.run.smithi164.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T16:13:58.828 INFO:teuthology.orchestra.run.smithi164.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:13:58.828 INFO:teuthology.orchestra.run.smithi164.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:13:58.828 INFO:teuthology.orchestra.run.smithi164.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:13:58.828 INFO:teuthology.orchestra.run.smithi164.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T16:13:58.828 INFO:teuthology.orchestra.run.smithi164.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T16:13:58.829 INFO:teuthology.orchestra.run.smithi164.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:13:58.829 INFO:teuthology.orchestra.run.smithi164.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:13:58.829 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T16:13:58.829 INFO:teuthology.orchestra.run.smithi164.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T16:13:58.830 INFO:teuthology.orchestra.run.smithi164.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:13:58.830 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T16:13:58.830 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T16:13:58.830 INFO:teuthology.orchestra.run.smithi164.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T16:13:58.830 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:13:58.831 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:13:58.831 INFO:teuthology.orchestra.run.smithi164.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:13:58.831 INFO:teuthology.orchestra.run.smithi164.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:13:58.831 INFO:teuthology.orchestra.run.smithi164.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:13:58.832 INFO:teuthology.orchestra.run.smithi164.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:13:58.832 INFO:teuthology.orchestra.run.smithi164.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:13:58.832 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:13:58.832 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.832 INFO:teuthology.orchestra.run.smithi164.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.833 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.833 INFO:teuthology.orchestra.run.smithi164.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:13:58.833 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:13:58.833 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:13:58.833 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:13:58.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:13:58.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:13:58.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:13:58.834 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:13:58.835 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:13:58.835 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:13:58.835 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:13:58.835 INFO:teuthology.orchestra.run.smithi164.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:13:58.836 INFO:teuthology.orchestra.run.smithi164.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:13:58.836 INFO:teuthology.orchestra.run.smithi164.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:13:58.836 INFO:teuthology.orchestra.run.smithi164.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:13:58.836 INFO:teuthology.orchestra.run.smithi164.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:13:58.836 INFO:teuthology.orchestra.run.smithi164.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:13:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:13:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:13:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:13:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:13:58.837 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:13:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:13:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:13:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:13:58.838 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:13:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:13:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:13:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:13:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.839 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:13:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:13:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:13:58.840 INFO:teuthology.orchestra.run.smithi164.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:13:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T16:13:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:13:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:13:58.841 INFO:teuthology.orchestra.run.smithi164.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:13:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:13:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:13:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:13:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:13:58.842 INFO:teuthology.orchestra.run.smithi164.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:13:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:13:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:13:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.843 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:13:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:13:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:13:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:13:58.844 INFO:teuthology.orchestra.run.smithi164.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:13:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:13:58.845 INFO:teuthology.orchestra.run.smithi164.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T16:13:58.846 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:13:58.846 INFO:teuthology.orchestra.run.smithi164.stdout:Complete! 2022-11-27T16:13:58.997 DEBUG:teuthology.parallel:result is None 2022-11-27T16:14:00.749 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:14:01.022 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:14:01.039 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:14:01.347 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T16:14:01.579 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T16:14:01.878 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T16:14:02.267 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T16:14:02.657 INFO:teuthology.orchestra.run.smithi167.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T16:14:02.904 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T16:14:02.979 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T16:14:03.187 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T16:14:03.200 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T16:14:03.379 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T16:14:03.387 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T16:14:03.516 INFO:teuthology.orchestra.run.smithi167.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-11-27T16:14:03.731 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:14:03.732 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:14:03.949 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:14:03.950 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:14:04.190 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:14:04.191 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:14:04.489 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:14:04.678 INFO:teuthology.orchestra.run.smithi167.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:14:05.235 INFO:teuthology.orchestra.run.smithi167.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:14:05.235 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T16:14:05.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T16:14:05.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T16:14:05.236 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T16:14:05.237 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T16:14:05.237 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T16:14:05.237 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:14:05.238 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:14:05.239 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:14:05.240 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:14:05.240 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T16:14:05.240 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T16:14:05.240 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T16:14:05.241 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T16:14:05.241 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:14:05.241 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T16:14:05.241 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T16:14:05.241 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T16:14:05.242 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T16:14:05.242 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T16:14:05.242 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T16:14:05.242 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T16:14:05.242 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T16:14:05.243 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T16:14:05.243 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T16:14:05.243 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T16:14:05.243 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T16:14:05.244 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T16:14:05.244 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T16:14:05.244 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T16:14:05.255 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T16:14:05.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T16:14:05.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T16:14:05.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T16:14:05.256 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T16:14:05.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T16:14:05.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T16:14:05.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T16:14:05.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T16:14:05.257 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T16:14:05.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T16:14:05.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T16:14:05.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T16:14:05.258 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T16:14:05.259 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T16:14:05.259 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T16:14:05.259 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T16:14:05.259 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T16:14:05.259 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T16:14:05.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T16:14:05.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T16:14:05.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T16:14:05.260 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T16:14:05.261 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T16:14:05.261 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T16:14:05.261 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T16:14:05.261 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T16:14:05.261 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T16:14:05.262 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T16:14:05.262 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T16:14:05.262 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T16:14:05.262 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T16:14:05.262 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T16:14:05.263 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T16:14:05.263 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T16:14:05.263 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T16:14:05.263 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T16:14:05.264 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T16:14:05.264 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:14:05.264 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T16:14:05.264 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T16:14:05.264 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-11-27T16:14:05.265 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-11-27T16:14:05.265 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-11-27T16:14:05.265 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-11-27T16:14:05.265 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-11-27T16:14:05.265 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-11-27T16:14:05.266 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-11-27T16:14:05.266 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-11-27T16:14:05.266 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-11-27T16:14:05.266 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-11-27T16:14:05.267 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-11-27T16:14:05.267 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-11-27T16:14:05.267 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-11-27T16:14:05.267 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-11-27T16:14:05.267 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-11-27T16:14:05.268 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-11-27T16:14:05.268 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-11-27T16:14:05.268 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-11-27T16:14:05.268 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-11-27T16:14:05.268 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-11-27T16:14:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-11-27T16:14:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-11-27T16:14:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-11-27T16:14:05.269 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-11-27T16:14:05.270 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-11-27T16:14:05.270 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-11-27T16:14:05.270 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-11-27T16:14:05.270 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-11-27T16:14:05.271 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-11-27T16:14:05.271 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-11-27T16:14:05.271 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-11-27T16:14:05.271 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-11-27T16:14:05.271 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-11-27T16:14:05.272 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-11-27T16:14:05.272 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-11-27T16:14:05.272 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-11-27T16:14:05.272 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-11-27T16:14:05.272 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-11-27T16:14:05.273 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-11-27T16:14:05.273 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-11-27T16:14:05.273 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-11-27T16:14:05.273 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-11-27T16:14:05.274 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-11-27T16:14:05.274 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-11-27T16:14:05.274 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-11-27T16:14:05.274 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-11-27T16:14:05.274 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-11-27T16:14:05.275 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-11-27T16:14:05.275 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-11-27T16:14:05.275 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-11-27T16:14:05.275 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-11-27T16:14:05.276 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-11-27T16:14:05.276 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-11-27T16:14:05.276 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-11-27T16:14:05.276 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-11-27T16:14:05.276 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-11-27T16:14:05.277 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-11-27T16:14:05.277 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T16:14:05.277 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T16:14:05.277 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T16:14:05.277 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T16:14:05.278 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T16:14:05.278 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:14:05.278 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T16:14:05.278 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T16:14:05.279 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T16:14:07.382 INFO:teuthology.orchestra.run.smithi167.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:14:07.382 INFO:teuthology.orchestra.run.smithi167.stdout:Installed products updated. 2022-11-27T16:14:07.643 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:07.643 INFO:teuthology.orchestra.run.smithi167.stdout:Upgraded: 2022-11-27T16:14:07.644 INFO:teuthology.orchestra.run.smithi167.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T16:14:07.644 INFO:teuthology.orchestra.run.smithi167.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.644 INFO:teuthology.orchestra.run.smithi167.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.645 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:14:07.645 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:14:07.646 INFO:teuthology.orchestra.run.smithi167.stdout:Installed: 2022-11-27T16:14:07.646 INFO:teuthology.orchestra.run.smithi167.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T16:14:07.646 INFO:teuthology.orchestra.run.smithi167.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:14:07.646 INFO:teuthology.orchestra.run.smithi167.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:14:07.646 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.647 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.648 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.649 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.649 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.649 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.649 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.650 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.650 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.650 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.650 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.651 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.651 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.651 INFO:teuthology.orchestra.run.smithi167.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.651 INFO:teuthology.orchestra.run.smithi167.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.652 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.652 INFO:teuthology.orchestra.run.smithi167.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:14:07.652 INFO:teuthology.orchestra.run.smithi167.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T16:14:07.652 INFO:teuthology.orchestra.run.smithi167.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:14:07.653 INFO:teuthology.orchestra.run.smithi167.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T16:14:07.653 INFO:teuthology.orchestra.run.smithi167.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:14:07.653 INFO:teuthology.orchestra.run.smithi167.stdout: jq-1.6-3.el8.x86_64 2022-11-27T16:14:07.653 INFO:teuthology.orchestra.run.smithi167.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:14:07.654 INFO:teuthology.orchestra.run.smithi167.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T16:14:07.654 INFO:teuthology.orchestra.run.smithi167.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:14:07.654 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.654 INFO:teuthology.orchestra.run.smithi167.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.655 INFO:teuthology.orchestra.run.smithi167.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.655 INFO:teuthology.orchestra.run.smithi167.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T16:14:07.655 INFO:teuthology.orchestra.run.smithi167.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:14:07.656 INFO:teuthology.orchestra.run.smithi167.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:14:07.656 INFO:teuthology.orchestra.run.smithi167.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:14:07.656 INFO:teuthology.orchestra.run.smithi167.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:14:07.656 INFO:teuthology.orchestra.run.smithi167.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:14:07.657 INFO:teuthology.orchestra.run.smithi167.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.657 INFO:teuthology.orchestra.run.smithi167.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.657 INFO:teuthology.orchestra.run.smithi167.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:14:07.657 INFO:teuthology.orchestra.run.smithi167.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.658 INFO:teuthology.orchestra.run.smithi167.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T16:14:07.658 INFO:teuthology.orchestra.run.smithi167.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:14:07.658 INFO:teuthology.orchestra.run.smithi167.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:14:07.658 INFO:teuthology.orchestra.run.smithi167.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:14:07.659 INFO:teuthology.orchestra.run.smithi167.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T16:14:07.659 INFO:teuthology.orchestra.run.smithi167.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T16:14:07.659 INFO:teuthology.orchestra.run.smithi167.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:14:07.660 INFO:teuthology.orchestra.run.smithi167.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:14:07.660 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T16:14:07.660 INFO:teuthology.orchestra.run.smithi167.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T16:14:07.660 INFO:teuthology.orchestra.run.smithi167.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:14:07.660 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T16:14:07.661 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T16:14:07.661 INFO:teuthology.orchestra.run.smithi167.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T16:14:07.661 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:14:07.661 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:14:07.662 INFO:teuthology.orchestra.run.smithi167.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:14:07.662 INFO:teuthology.orchestra.run.smithi167.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:14:07.662 INFO:teuthology.orchestra.run.smithi167.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:14:07.662 INFO:teuthology.orchestra.run.smithi167.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:14:07.663 INFO:teuthology.orchestra.run.smithi167.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:14:07.663 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:14:07.663 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.663 INFO:teuthology.orchestra.run.smithi167.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.664 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.664 INFO:teuthology.orchestra.run.smithi167.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:14:07.664 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:14:07.664 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:14:07.665 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:14:07.665 INFO:teuthology.orchestra.run.smithi167.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:14:07.665 INFO:teuthology.orchestra.run.smithi167.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:14:07.665 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:14:07.666 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:14:07.666 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:14:07.666 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:14:07.667 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:14:07.667 INFO:teuthology.orchestra.run.smithi167.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:14:07.668 INFO:teuthology.orchestra.run.smithi167.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:14:07.668 INFO:teuthology.orchestra.run.smithi167.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:14:07.669 INFO:teuthology.orchestra.run.smithi167.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:14:07.669 INFO:teuthology.orchestra.run.smithi167.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:14:07.670 INFO:teuthology.orchestra.run.smithi167.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:14:07.670 INFO:teuthology.orchestra.run.smithi167.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:14:07.670 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:14:07.671 INFO:teuthology.orchestra.run.smithi167.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:14:07.671 INFO:teuthology.orchestra.run.smithi167.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:14:07.672 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:14:07.672 INFO:teuthology.orchestra.run.smithi167.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:14:07.673 INFO:teuthology.orchestra.run.smithi167.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:14:07.673 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:14:07.674 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:14:07.674 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:14:07.675 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:14:07.675 INFO:teuthology.orchestra.run.smithi167.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:14:07.676 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.676 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.677 INFO:teuthology.orchestra.run.smithi167.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:14:07.677 INFO:teuthology.orchestra.run.smithi167.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:14:07.677 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.678 INFO:teuthology.orchestra.run.smithi167.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:14:07.678 INFO:teuthology.orchestra.run.smithi167.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:14:07.679 INFO:teuthology.orchestra.run.smithi167.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T16:14:07.679 INFO:teuthology.orchestra.run.smithi167.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:14:07.680 INFO:teuthology.orchestra.run.smithi167.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:14:07.680 INFO:teuthology.orchestra.run.smithi167.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:14:07.681 INFO:teuthology.orchestra.run.smithi167.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:14:07.681 INFO:teuthology.orchestra.run.smithi167.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:14:07.681 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:14:07.682 INFO:teuthology.orchestra.run.smithi167.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:14:07.682 INFO:teuthology.orchestra.run.smithi167.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:14:07.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:14:07.683 INFO:teuthology.orchestra.run.smithi167.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:14:07.684 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.684 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.685 INFO:teuthology.orchestra.run.smithi167.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:14:07.685 INFO:teuthology.orchestra.run.smithi167.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:14:07.686 INFO:teuthology.orchestra.run.smithi167.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:14:07.686 INFO:teuthology.orchestra.run.smithi167.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:14:07.687 INFO:teuthology.orchestra.run.smithi167.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:14:07.687 INFO:teuthology.orchestra.run.smithi167.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:14:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T16:14:07.689 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:07.690 INFO:teuthology.orchestra.run.smithi167.stdout:Complete! 2022-11-27T16:14:07.832 DEBUG:teuthology.parallel:result is None 2022-11-27T16:14:07.834 DEBUG:teuthology.orchestra.run.smithi164:> sudo lsb_release -is 2022-11-27T16:14:07.900 INFO:teuthology.orchestra.run.smithi164.stdout:RedHatEnterprise 2022-11-27T16:14:07.902 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:14:07.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:14:08.087 DEBUG:teuthology.orchestra.run.smithi164:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:14:08.112 INFO:teuthology.orchestra.run.smithi164.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:14:08.113 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:14:08.113 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:14:08.114 DEBUG:teuthology.orchestra.run.smithi167:> sudo lsb_release -is 2022-11-27T16:14:08.171 INFO:teuthology.orchestra.run.smithi167.stdout:RedHatEnterprise 2022-11-27T16:14:08.172 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:14:08.173 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:14:08.326 DEBUG:teuthology.orchestra.run.smithi167:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:14:08.349 INFO:teuthology.orchestra.run.smithi167.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:14:08.349 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:14:08.349 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:14:08.350 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T16:14:08.351 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:08.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:14:08.383 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:08.383 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:14:08.414 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T16:14:08.415 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:08.416 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:14:08.454 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:14:08.524 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:08.524 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:14:08.551 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:14:08.618 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T16:14:08.618 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:08.619 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:14:08.651 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:14:08.720 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:08.720 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:14:08.748 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:14:08.813 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T16:14:08.884 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T16:14:08.885 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 777 /var/log/ceph 2022-11-27T16:14:08.888 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 777 /var/log/ceph 2022-11-27T16:14:08.915 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T16:14:08.915 DEBUG:teuthology.orchestra.run.smithi164:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:14:08.959 DEBUG:teuthology.orchestra.run.smithi167:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:14:08.986 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T16:14:08.988 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:14:09.027 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:14:09.091 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-11-27T16:14:09.091 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-11-27T16:14:09.092 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-11-27T16:14:09.092 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:09.092 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:14:09.110 DEBUG:teuthology.orchestra.run.smithi164:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:14:09.232 DEBUG:teuthology.orchestra.run.smithi164:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:14:09.260 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:09.260 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:14:09.278 DEBUG:teuthology.orchestra.run.smithi167:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:14:09.393 DEBUG:teuthology.orchestra.run.smithi167:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:14:09.426 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T16:14:09.427 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T16:14:09.427 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088642', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/damage}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088642', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'Error loading MDS rank', 'missing journal object', 'Error recovering journal', 'error decoding table object', 'failed to read JournalPointer', 'Corrupt directory entry', 'Corrupt fnode header', 'corrupt sessionmap header', 'Corrupt dentry', 'Scrub error on inode', 'Metadata damage detected', 'MDS_READ_ONLY', 'force file system read-only'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi164.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE4l3lGZ5h24xE8pTYzNyzYudEsCeJp+6ANFj26XmwfbbTyGUCCkPBXVnC4WsYtIdJWJ+CesbUzristFq8d4svrICJ29kvk4Sqtnc1MQ3ERMchxvytrwizMNa3m/DJDfQsKWYMZDfaQtiQPONidWkPtfJUqaG3DA9oqfniz6XFSjRT8AILrg6yzWosq0wUdFGb6XKp9pWCmZrntPfZ7avWKpb29gdQJsaUOi2saj/iowMhii0hi+fKAickbk+TnHKKxfRHLK9tRXuUEqIas7sv31/kGQlk4Tg3gcpjBSAh03Ou+lUQV6ckKoLiEgmQFbTDxWW/XVwXNqJJDCOXNZKVkyp6CGX3h7TiIUl0hcS1TTPoVfl7NjHG8TpLy4ON8yKuo+jjeDoTcvpER/0ZpQeuNlOsoJtHakMFcIB2Q7XSK9U5wTSa90PtsW9i4mE2jJN8pyBptiB2wKg4zaCBefMhWXwwQjTvK0ATgCdyKP7uI092Di8mkU73uMlI5xbR5YM=', 'smithi167.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCi7ulleg3cczC64eNJF3widiQB3AdECOlLOwZ1Q9YPG17ivA3eiSfOV+bvZURS2laJZzijIYgU9q7aNOkso89rp/6q4R2QdFFelHCRuOVWZfTDjoR66oQXlHYloo0ZtwcH0za5MAYWEY6Krqgq20qqJ8TAiSWdTXSg1ZbS+AmWi8uYYM91HIPukS/ch9RUGOaHWp+oHzRIHKYI+sNftvLuS4N7PdO3w9nq4HkgBJOsv/9WUFIWlpWBQph25aVkZDWddMdLTQFPK2zeF3ep3TaNg2Uwv7I1Z5zhpjkX2BwV6bkL78d6V38rPUuvCCcBqqHM8gIsnQ6+OsnJiwXROPTH1Syb44O9836AkCEiIHoReQXw2GhuPCqNTMMicuqxF1RhHYnge4fB/ZpqQraVIZo03XYAAK+JN1Ir8lWL9VzFGIz4GT0NaTZVwHd/dbes2XOeZ04sEkBgVZFXMbDBjJoCkevZOFCBiGclvpWAdTZSNKWczvpzroVvRtF/y8DYjdc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_damage']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.23126'} 2022-11-27T16:14:09.428 DEBUG:teuthology.orchestra.run.smithi164:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:14:09.431 DEBUG:teuthology.orchestra.run.smithi167:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:14:09.478 DEBUG:teuthology.orchestra.run.smithi164:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:14:09.488 DEBUG:teuthology.orchestra.run.smithi167:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:14:09.545 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:09.545 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:14:09.573 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:14:09.573 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_1 2022-11-27T16:14:09.630 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:14:09.631 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:09.631 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 366662 Links: 1 2022-11-27T16:14:09.631 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:09.632 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:09.632 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2022-11-27 16:13:56.155675723 +0000 2022-11-27T16:14:09.632 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2022-11-27 16:11:13.331841004 +0000 2022-11-27T16:14:09.632 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2022-11-27 16:11:13.331841004 +0000 2022-11-27T16:14:09.632 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2022-11-27T16:14:09.633 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:14:09.696 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2022-11-27T16:14:09.697 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2022-11-27T16:14:09.697 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000218232 s, 2.3 MB/s 2022-11-27T16:14:09.699 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:14:09.758 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_2 2022-11-27T16:14:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:14:09.819 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:09.820 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 367628 Links: 1 2022-11-27T16:14:09.820 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:09.820 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:09.821 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2022-11-27 16:13:56.155675723 +0000 2022-11-27T16:14:09.821 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2022-11-27 16:11:13.877832323 +0000 2022-11-27T16:14:09.821 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2022-11-27 16:11:13.877832323 +0000 2022-11-27T16:14:09.822 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2022-11-27T16:14:09.822 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:14:09.884 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2022-11-27T16:14:09.885 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2022-11-27T16:14:09.885 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000187192 s, 2.7 MB/s 2022-11-27T16:14:09.887 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:14:09.945 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_3 2022-11-27T16:14:10.003 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:14:10.004 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:10.004 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 361392 Links: 1 2022-11-27T16:14:10.005 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:10.005 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:10.006 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2022-11-27 16:13:56.155675723 +0000 2022-11-27T16:14:10.006 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2022-11-27 16:11:14.471822879 +0000 2022-11-27T16:14:10.007 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2022-11-27 16:11:14.471822879 +0000 2022-11-27T16:14:10.007 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2022-11-27T16:14:10.008 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:14:10.068 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2022-11-27T16:14:10.069 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2022-11-27T16:14:10.070 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.00022602 s, 2.3 MB/s 2022-11-27T16:14:10.073 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:14:10.131 DEBUG:teuthology.orchestra.run.smithi164:> stat /dev/vg_nvme/lv_4 2022-11-27T16:14:10.188 INFO:teuthology.orchestra.run.smithi164.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:14:10.189 INFO:teuthology.orchestra.run.smithi164.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:10.189 INFO:teuthology.orchestra.run.smithi164.stdout:Device: 6h/6d Inode: 370711 Links: 1 2022-11-27T16:14:10.189 INFO:teuthology.orchestra.run.smithi164.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:10.189 INFO:teuthology.orchestra.run.smithi164.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:10.190 INFO:teuthology.orchestra.run.smithi164.stdout:Access: 2022-11-27 16:13:56.155675723 +0000 2022-11-27T16:14:10.190 INFO:teuthology.orchestra.run.smithi164.stdout:Modify: 2022-11-27 16:11:15.042813800 +0000 2022-11-27T16:14:10.190 INFO:teuthology.orchestra.run.smithi164.stdout:Change: 2022-11-27 16:11:15.042813800 +0000 2022-11-27T16:14:10.190 INFO:teuthology.orchestra.run.smithi164.stdout: Birth: - 2022-11-27T16:14:10.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:14:10.253 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records in 2022-11-27T16:14:10.253 INFO:teuthology.orchestra.run.smithi164.stderr:1+0 records out 2022-11-27T16:14:10.253 INFO:teuthology.orchestra.run.smithi164.stderr:512 bytes copied, 0.000196264 s, 2.6 MB/s 2022-11-27T16:14:10.255 DEBUG:teuthology.orchestra.run.smithi164:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:14:10.313 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:10.313 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:10.314 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:14:10.333 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:14:10.334 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_1 2022-11-27T16:14:10.392 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:14:10.393 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:10.393 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 359896 Links: 1 2022-11-27T16:14:10.394 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:10.394 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:10.395 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2022-11-27 16:14:04.930578496 +0000 2022-11-27T16:14:10.395 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2022-11-27 16:11:13.337692681 +0000 2022-11-27T16:14:10.396 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2022-11-27 16:11:13.337692681 +0000 2022-11-27T16:14:10.396 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2022-11-27T16:14:10.397 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:14:10.459 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2022-11-27T16:14:10.459 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2022-11-27T16:14:10.460 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000228461 s, 2.2 MB/s 2022-11-27T16:14:10.461 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:14:10.520 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_2 2022-11-27T16:14:10.578 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:14:10.579 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:10.579 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 365825 Links: 1 2022-11-27T16:14:10.579 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:10.579 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:10.580 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2022-11-27 16:14:04.930578496 +0000 2022-11-27T16:14:10.580 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2022-11-27 16:11:13.872683024 +0000 2022-11-27T16:14:10.580 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2022-11-27 16:11:13.872683024 +0000 2022-11-27T16:14:10.580 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2022-11-27T16:14:10.581 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:14:10.644 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2022-11-27T16:14:10.644 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2022-11-27T16:14:10.645 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00021557 s, 2.4 MB/s 2022-11-27T16:14:10.646 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:14:10.704 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_3 2022-11-27T16:14:10.761 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:14:10.762 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:10.762 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 359995 Links: 1 2022-11-27T16:14:10.762 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2022-11-27 16:14:04.930578496 +0000 2022-11-27T16:14:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2022-11-27 16:11:14.453672538 +0000 2022-11-27T16:14:10.763 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2022-11-27 16:11:14.453672538 +0000 2022-11-27T16:14:10.764 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2022-11-27T16:14:10.764 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:14:10.827 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2022-11-27T16:14:10.827 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2022-11-27T16:14:10.827 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.000206233 s, 2.5 MB/s 2022-11-27T16:14:10.829 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:14:10.886 DEBUG:teuthology.orchestra.run.smithi167:> stat /dev/vg_nvme/lv_4 2022-11-27T16:14:10.943 INFO:teuthology.orchestra.run.smithi167.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:14:10.944 INFO:teuthology.orchestra.run.smithi167.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:14:10.944 INFO:teuthology.orchestra.run.smithi167.stdout:Device: 6h/6d Inode: 361260 Links: 1 2022-11-27T16:14:10.944 INFO:teuthology.orchestra.run.smithi167.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:14:10.944 INFO:teuthology.orchestra.run.smithi167.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:14:10.944 INFO:teuthology.orchestra.run.smithi167.stdout:Access: 2022-11-27 16:14:04.930578496 +0000 2022-11-27T16:14:10.945 INFO:teuthology.orchestra.run.smithi167.stdout:Modify: 2022-11-27 16:11:15.021662286 +0000 2022-11-27T16:14:10.945 INFO:teuthology.orchestra.run.smithi167.stdout:Change: 2022-11-27 16:11:15.021662286 +0000 2022-11-27T16:14:10.945 INFO:teuthology.orchestra.run.smithi167.stdout: Birth: - 2022-11-27T16:14:10.945 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:14:11.009 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records in 2022-11-27T16:14:11.009 INFO:teuthology.orchestra.run.smithi167.stderr:1+0 records out 2022-11-27T16:14:11.009 INFO:teuthology.orchestra.run.smithi167.stderr:512 bytes copied, 0.00022077 s, 2.3 MB/s 2022-11-27T16:14:11.011 DEBUG:teuthology.orchestra.run.smithi167:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:14:11.070 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:11.071 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:14:11.071 INFO:tasks.ceph:Generating config... 2022-11-27T16:14:11.075 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T16:14:11.076 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T16:14:11.077 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T16:14:11.077 INFO:tasks.ceph:[client] pid file = 2022-11-27T16:14:11.078 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T16:14:11.078 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T16:14:11.079 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T16:14:11.079 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T16:14:11.080 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T16:14:11.081 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T16:14:11.081 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T16:14:11.082 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T16:14:11.082 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T16:14:11.083 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T16:14:11.083 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T16:14:11.084 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T16:14:11.085 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T16:14:11.085 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T16:14:11.085 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T16:14:11.086 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T16:14:11.086 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T16:14:11.086 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T16:14:11.087 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T16:14:11.087 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T16:14:11.087 INFO:tasks.ceph:[osd] debug bluefs = 20 2022-11-27T16:14:11.088 INFO:tasks.ceph:[osd] debug bluestore = 20 2022-11-27T16:14:11.088 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T16:14:11.088 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T16:14:11.088 INFO:tasks.ceph:[osd] debug rocksdb = 10 2022-11-27T16:14:11.089 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T16:14:11.089 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T16:14:11.089 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T16:14:11.090 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2022-11-27T16:14:11.090 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2022-11-27T16:14:11.090 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T16:14:11.091 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T16:14:11.091 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T16:14:11.173 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T16:14:11.176 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T16:14:11.225 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:14:11.252 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.164'), ('mon.b', '172.21.15.167'), ('mon.c', '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]')] 2022-11-27T16:14:11.253 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.164,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T16:14:11.255 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:11.255 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:14:11.310 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.164 --add b 172.21.15.167 --addv c '[v2:172.21.15.167:3301,v1:172.21.15.167:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:11.399 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:11.400 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: generated fsid 94c9dd9b-1805-4f9e-833a-3b29514744e3 2022-11-27T16:14:11.400 INFO:teuthology.orchestra.run.smithi164.stdout:setting min_mon_release = pacific 2022-11-27T16:14:11.400 INFO:teuthology.orchestra.run.smithi164.stdout:epoch 0 2022-11-27T16:14:11.400 INFO:teuthology.orchestra.run.smithi164.stdout:fsid 94c9dd9b-1805-4f9e-833a-3b29514744e3 2022-11-27T16:14:11.401 INFO:teuthology.orchestra.run.smithi164.stdout:last_changed 2022-11-27T16:14:11.398598+0000 2022-11-27T16:14:11.401 INFO:teuthology.orchestra.run.smithi164.stdout:created 2022-11-27T16:14:11.398598+0000 2022-11-27T16:14:11.401 INFO:teuthology.orchestra.run.smithi164.stdout:min_mon_release 16 (pacific) 2022-11-27T16:14:11.401 INFO:teuthology.orchestra.run.smithi164.stdout:election_strategy: 1 2022-11-27T16:14:11.401 INFO:teuthology.orchestra.run.smithi164.stdout:0: [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] mon.a 2022-11-27T16:14:11.402 INFO:teuthology.orchestra.run.smithi164.stdout:1: [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] mon.b 2022-11-27T16:14:11.402 INFO:teuthology.orchestra.run.smithi164.stdout:2: [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] mon.c 2022-11-27T16:14:11.402 INFO:teuthology.orchestra.run.smithi164.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T16:14:11.404 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:14:11.420 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 94c9dd9b-1805-4f9e-833a-3b29514744e3... 2022-11-27T16:14:11.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:14:11.463 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:14:11.506 INFO:teuthology.orchestra.run.smithi164.stdout:[global] 2022-11-27T16:14:11.507 INFO:teuthology.orchestra.run.smithi164.stdout: chdir = "" 2022-11-27T16:14:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:14:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: auth supported = cephx 2022-11-27T16:14:11.508 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: filestore xattr use omap = true 2022-11-27T16:14:11.509 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: mon clock drift allowed = 1.000 2022-11-27T16:14:11.510 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: osd crush chooseleaf type = 0 2022-11-27T16:14:11.511 INFO:teuthology.orchestra.run.smithi164.stdout: auth debug = true 2022-11-27T16:14:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.512 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on old message = true 2022-11-27T16:14:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: ms die on bug = true 2022-11-27T16:14:11.513 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:14:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: mon pg warn max object skew = 0 2022-11-27T16:14:11.514 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:14:11.515 INFO:teuthology.orchestra.run.smithi164.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:14:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.516 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default size = 2 2022-11-27T16:14:11.517 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow primary affinity = true 2022-11-27T16:14:11.518 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd allow pg remap = true 2022-11-27T16:14:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:14:11.519 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:14:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on no sortbitwise = false 2022-11-27T16:14:11.520 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:14:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: mon warn on too few osds = false 2022-11-27T16:14:11.521 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:14:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:14:11.522 INFO:teuthology.orchestra.run.smithi164.stdout: mon_allow_pool_size_one = true 2022-11-27T16:14:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:14:11.523 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: osd default data pool replay window = 5 2022-11-27T16:14:11.524 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: mon allow pool delete = true 2022-11-27T16:14:11.525 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.526 INFO:teuthology.orchestra.run.smithi164.stdout: mon cluster log file level = debug 2022-11-27T16:14:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: debug asserts on shutdown = true 2022-11-27T16:14:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: mon health detail to clog = false 2022-11-27T16:14:11.527 INFO:teuthology.orchestra.run.smithi164.stdout: mon host = "172.21.15.164,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2022-11-27T16:14:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: fsid = 94c9dd9b-1805-4f9e-833a-3b29514744e3 2022-11-27T16:14:11.528 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.529 INFO:teuthology.orchestra.run.smithi164.stdout:[osd] 2022-11-27T16:14:11.529 INFO:teuthology.orchestra.run.smithi164.stdout: osd journal size = 100 2022-11-27T16:14:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.530 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub load threshold = 5.0 2022-11-27T16:14:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: osd scrub max interval = 600 2022-11-27T16:14:11.531 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: osd recover clone overlap = true 2022-11-27T16:14:11.532 INFO:teuthology.orchestra.run.smithi164.stdout: osd recovery max chunk = 1048576 2022-11-27T16:14:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.533 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug shutdown = true 2022-11-27T16:14:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug op order = true 2022-11-27T16:14:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug verify stray on activate = true 2022-11-27T16:14:11.534 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: osd open classes on start = true 2022-11-27T16:14:11.535 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug pg log writeout = true 2022-11-27T16:14:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.536 INFO:teuthology.orchestra.run.smithi164.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:14:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.537 INFO:teuthology.orchestra.run.smithi164.stdout: osd map max advance = 10 2022-11-27T16:14:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.538 INFO:teuthology.orchestra.run.smithi164.stdout: journal zero on create = true 2022-11-27T16:14:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.539 INFO:teuthology.orchestra.run.smithi164.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:14:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: filestore apply finisher threads = 3 2022-11-27T16:14:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.540 INFO:teuthology.orchestra.run.smithi164.stdout: bdev debug aio = true 2022-11-27T16:14:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: osd debug misdirected ops = true 2022-11-27T16:14:11.542 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore block size = 96636764160 2022-11-27T16:14:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: bluestore fsck on mount = True 2022-11-27T16:14:11.543 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluefs = 20 2022-11-27T16:14:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: debug bluestore = 20 2022-11-27T16:14:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-11-27T16:14:11.544 INFO:teuthology.orchestra.run.smithi164.stdout: debug osd = 20 2022-11-27T16:14:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: debug rocksdb = 10 2022-11-27T16:14:11.545 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:14:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd full ratio = 0.9 2022-11-27T16:14:11.546 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:14:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: osd failsafe full ratio = 0.95 2022-11-27T16:14:11.547 INFO:teuthology.orchestra.run.smithi164.stdout: osd objectstore = bluestore 2022-11-27T16:14:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-11-27T16:14:11.548 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.549 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr] 2022-11-27T16:14:11.549 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-11-27T16:14:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: debug mgr = 20 2022-11-27T16:14:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-11-27T16:14:11.550 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-11-27T16:14:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:14:11.551 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:14:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: mgr/telemetry/nag = false 2022-11-27T16:14:11.552 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.553 INFO:teuthology.orchestra.run.smithi164.stdout:[mon] 2022-11-27T16:14:11.553 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-11-27T16:14:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: debug mon = 20 2022-11-27T16:14:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: debug paxos = 20 2022-11-27T16:14:11.554 INFO:teuthology.orchestra.run.smithi164.stdout: debug auth = 20 2022-11-27T16:14:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: mon data avail warn = 5 2022-11-27T16:14:11.555 INFO:teuthology.orchestra.run.smithi164.stdout: mon mgr mkfs grace = 240 2022-11-27T16:14:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:14:11.556 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd reporter subtree level = osd 2022-11-27T16:14:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: mon osd prime pg temp = true 2022-11-27T16:14:11.557 INFO:teuthology.orchestra.run.smithi164.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:14:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.558 INFO:teuthology.orchestra.run.smithi164.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:14:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:14:11.559 INFO:teuthology.orchestra.run.smithi164.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:14:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.560 INFO:teuthology.orchestra.run.smithi164.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:14:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:14:11.561 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:14:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: # 1m isn't quite enough 2022-11-27T16:14:11.562 INFO:teuthology.orchestra.run.smithi164.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:14:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.563 INFO:teuthology.orchestra.run.smithi164.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:14:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: mon op complaint time = 120 2022-11-27T16:14:11.564 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:11.565 INFO:teuthology.orchestra.run.smithi164.stdout:[client] 2022-11-27T16:14:11.565 INFO:teuthology.orchestra.run.smithi164.stdout: rgw cache enabled = true 2022-11-27T16:14:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable ops log = true 2022-11-27T16:14:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: rgw enable usage log = true 2022-11-27T16:14:11.566 INFO:teuthology.orchestra.run.smithi164.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:14:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:14:11.567 INFO:teuthology.orchestra.run.smithi164.stdout: client mount timeout = 600 2022-11-27T16:14:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: debug client = 20 2022-11-27T16:14:11.568 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-11-27T16:14:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: pid file = "" 2022-11-27T16:14:11.569 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2022-11-27T16:14:11.570 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2022-11-27T16:14:11.570 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.a] 2022-11-27T16:14:11.571 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.b] 2022-11-27T16:14:11.571 INFO:teuthology.orchestra.run.smithi164.stdout:[mon.c] 2022-11-27T16:14:11.572 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2022-11-27T16:14:11.572 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2022-11-27T16:14:11.573 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2022-11-27T16:14:11.573 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2022-11-27T16:14:11.574 INFO:teuthology.orchestra.run.smithi164.stdout:[mds] 2022-11-27T16:14:11.574 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds = 20 2022-11-27T16:14:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: debug mds balancer = 20 2022-11-27T16:14:11.575 INFO:teuthology.orchestra.run.smithi164.stdout: debug ms = 1 2022-11-27T16:14:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug frag = True 2022-11-27T16:14:11.576 INFO:teuthology.orchestra.run.smithi164.stdout: mds debug scatterstat = True 2022-11-27T16:14:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: mds op complaint time = 180 2022-11-27T16:14:11.577 INFO:teuthology.orchestra.run.smithi164.stdout: mds verify scatter = True 2022-11-27T16:14:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: osd op complaint time = 180 2022-11-27T16:14:11.578 INFO:teuthology.orchestra.run.smithi164.stdout: rados mon op timeout = 900 2022-11-27T16:14:11.579 INFO:teuthology.orchestra.run.smithi164.stdout: rados osd op timeout = 900 2022-11-27T16:14:11.583 INFO:teuthology.orchestra.run.smithi167.stdout:[global] 2022-11-27T16:14:11.583 INFO:teuthology.orchestra.run.smithi167.stdout: chdir = "" 2022-11-27T16:14:11.584 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:14:11.584 INFO:teuthology.orchestra.run.smithi167.stdout: auth supported = cephx 2022-11-27T16:14:11.585 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.585 INFO:teuthology.orchestra.run.smithi167.stdout: filestore xattr use omap = true 2022-11-27T16:14:11.586 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.586 INFO:teuthology.orchestra.run.smithi167.stdout: mon clock drift allowed = 1.000 2022-11-27T16:14:11.587 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.587 INFO:teuthology.orchestra.run.smithi167.stdout: osd crush chooseleaf type = 0 2022-11-27T16:14:11.588 INFO:teuthology.orchestra.run.smithi167.stdout: auth debug = true 2022-11-27T16:14:11.588 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.589 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on old message = true 2022-11-27T16:14:11.589 INFO:teuthology.orchestra.run.smithi167.stdout: ms die on bug = true 2022-11-27T16:14:11.590 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.590 INFO:teuthology.orchestra.run.smithi167.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:14:11.590 INFO:teuthology.orchestra.run.smithi167.stdout: mon pg warn max object skew = 0 2022-11-27T16:14:11.591 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.591 INFO:teuthology.orchestra.run.smithi167.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:14:11.592 INFO:teuthology.orchestra.run.smithi167.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:14:11.592 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.593 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default size = 2 2022-11-27T16:14:11.593 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.594 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow primary affinity = true 2022-11-27T16:14:11.594 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd allow pg remap = true 2022-11-27T16:14:11.595 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:14:11.595 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:14:11.596 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on no sortbitwise = false 2022-11-27T16:14:11.596 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:14:11.597 INFO:teuthology.orchestra.run.smithi167.stdout: mon warn on too few osds = false 2022-11-27T16:14:11.597 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:14:11.598 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:14:11.598 INFO:teuthology.orchestra.run.smithi167.stdout: mon_allow_pool_size_one = true 2022-11-27T16:14:11.599 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.599 INFO:teuthology.orchestra.run.smithi167.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:14:11.600 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.600 INFO:teuthology.orchestra.run.smithi167.stdout: osd default data pool replay window = 5 2022-11-27T16:14:11.600 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.600 INFO:teuthology.orchestra.run.smithi167.stdout: mon allow pool delete = true 2022-11-27T16:14:11.601 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.601 INFO:teuthology.orchestra.run.smithi167.stdout: mon cluster log file level = debug 2022-11-27T16:14:11.601 INFO:teuthology.orchestra.run.smithi167.stdout: debug asserts on shutdown = true 2022-11-27T16:14:11.601 INFO:teuthology.orchestra.run.smithi167.stdout: mon health detail to clog = false 2022-11-27T16:14:11.601 INFO:teuthology.orchestra.run.smithi167.stdout: mon host = "172.21.15.164,172.21.15.167,[v2:172.21.15.167:3301,v1:172.21.15.167:6790]" 2022-11-27T16:14:11.602 INFO:teuthology.orchestra.run.smithi167.stdout: fsid = 94c9dd9b-1805-4f9e-833a-3b29514744e3 2022-11-27T16:14:11.602 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.603 INFO:teuthology.orchestra.run.smithi167.stdout:[osd] 2022-11-27T16:14:11.603 INFO:teuthology.orchestra.run.smithi167.stdout: osd journal size = 100 2022-11-27T16:14:11.603 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.603 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub load threshold = 5.0 2022-11-27T16:14:11.604 INFO:teuthology.orchestra.run.smithi167.stdout: osd scrub max interval = 600 2022-11-27T16:14:11.604 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.604 INFO:teuthology.orchestra.run.smithi167.stdout: osd recover clone overlap = true 2022-11-27T16:14:11.605 INFO:teuthology.orchestra.run.smithi167.stdout: osd recovery max chunk = 1048576 2022-11-27T16:14:11.605 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.605 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug shutdown = true 2022-11-27T16:14:11.605 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug op order = true 2022-11-27T16:14:11.606 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug verify stray on activate = true 2022-11-27T16:14:11.606 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.606 INFO:teuthology.orchestra.run.smithi167.stdout: osd open classes on start = true 2022-11-27T16:14:11.606 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug pg log writeout = true 2022-11-27T16:14:11.607 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.607 INFO:teuthology.orchestra.run.smithi167.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:14:11.607 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.607 INFO:teuthology.orchestra.run.smithi167.stdout: osd map max advance = 10 2022-11-27T16:14:11.608 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.608 INFO:teuthology.orchestra.run.smithi167.stdout: journal zero on create = true 2022-11-27T16:14:11.608 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.608 INFO:teuthology.orchestra.run.smithi167.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:14:11.609 INFO:teuthology.orchestra.run.smithi167.stdout: filestore apply finisher threads = 3 2022-11-27T16:14:11.609 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.609 INFO:teuthology.orchestra.run.smithi167.stdout: bdev debug aio = true 2022-11-27T16:14:11.609 INFO:teuthology.orchestra.run.smithi167.stdout: osd debug misdirected ops = true 2022-11-27T16:14:11.609 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore block size = 96636764160 2022-11-27T16:14:11.610 INFO:teuthology.orchestra.run.smithi167.stdout: bluestore fsck on mount = True 2022-11-27T16:14:11.610 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluefs = 20 2022-11-27T16:14:11.610 INFO:teuthology.orchestra.run.smithi167.stdout: debug bluestore = 20 2022-11-27T16:14:11.610 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2022-11-27T16:14:11.610 INFO:teuthology.orchestra.run.smithi167.stdout: debug osd = 20 2022-11-27T16:14:11.611 INFO:teuthology.orchestra.run.smithi167.stdout: debug rocksdb = 10 2022-11-27T16:14:11.611 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:14:11.611 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd full ratio = 0.9 2022-11-27T16:14:11.612 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:14:11.612 INFO:teuthology.orchestra.run.smithi167.stdout: osd failsafe full ratio = 0.95 2022-11-27T16:14:11.612 INFO:teuthology.orchestra.run.smithi167.stdout: osd objectstore = bluestore 2022-11-27T16:14:11.613 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2022-11-27T16:14:11.613 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.614 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr] 2022-11-27T16:14:11.614 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2022-11-27T16:14:11.614 INFO:teuthology.orchestra.run.smithi167.stdout: debug mgr = 20 2022-11-27T16:14:11.615 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2022-11-27T16:14:11.615 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2022-11-27T16:14:11.615 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:14:11.616 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:14:11.616 INFO:teuthology.orchestra.run.smithi167.stdout: mgr/telemetry/nag = false 2022-11-27T16:14:11.616 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.617 INFO:teuthology.orchestra.run.smithi167.stdout:[mon] 2022-11-27T16:14:11.617 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2022-11-27T16:14:11.618 INFO:teuthology.orchestra.run.smithi167.stdout: debug mon = 20 2022-11-27T16:14:11.619 INFO:teuthology.orchestra.run.smithi167.stdout: debug paxos = 20 2022-11-27T16:14:11.619 INFO:teuthology.orchestra.run.smithi167.stdout: debug auth = 20 2022-11-27T16:14:11.619 INFO:teuthology.orchestra.run.smithi167.stdout: mon data avail warn = 5 2022-11-27T16:14:11.619 INFO:teuthology.orchestra.run.smithi167.stdout: mon mgr mkfs grace = 240 2022-11-27T16:14:11.620 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:14:11.620 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd reporter subtree level = osd 2022-11-27T16:14:11.620 INFO:teuthology.orchestra.run.smithi167.stdout: mon osd prime pg temp = true 2022-11-27T16:14:11.620 INFO:teuthology.orchestra.run.smithi167.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:14:11.621 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.621 INFO:teuthology.orchestra.run.smithi167.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:14:11.621 INFO:teuthology.orchestra.run.smithi167.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:14:11.621 INFO:teuthology.orchestra.run.smithi167.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:14:11.621 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.622 INFO:teuthology.orchestra.run.smithi167.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:14:11.622 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:14:11.622 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:14:11.622 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.623 INFO:teuthology.orchestra.run.smithi167.stdout: # 1m isn't quite enough 2022-11-27T16:14:11.623 INFO:teuthology.orchestra.run.smithi167.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:14:11.623 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.624 INFO:teuthology.orchestra.run.smithi167.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:14:11.624 INFO:teuthology.orchestra.run.smithi167.stdout: mon op complaint time = 120 2022-11-27T16:14:11.624 INFO:teuthology.orchestra.run.smithi167.stdout: 2022-11-27T16:14:11.624 INFO:teuthology.orchestra.run.smithi167.stdout:[client] 2022-11-27T16:14:11.625 INFO:teuthology.orchestra.run.smithi167.stdout: rgw cache enabled = true 2022-11-27T16:14:11.625 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable ops log = true 2022-11-27T16:14:11.626 INFO:teuthology.orchestra.run.smithi167.stdout: rgw enable usage log = true 2022-11-27T16:14:11.626 INFO:teuthology.orchestra.run.smithi167.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:14:11.626 INFO:teuthology.orchestra.run.smithi167.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:14:11.626 INFO:teuthology.orchestra.run.smithi167.stdout: client mount timeout = 600 2022-11-27T16:14:11.627 INFO:teuthology.orchestra.run.smithi167.stdout: debug client = 20 2022-11-27T16:14:11.627 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2022-11-27T16:14:11.627 INFO:teuthology.orchestra.run.smithi167.stdout: pid file = "" 2022-11-27T16:14:11.627 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2022-11-27T16:14:11.628 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2022-11-27T16:14:11.628 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.a] 2022-11-27T16:14:11.628 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.b] 2022-11-27T16:14:11.628 INFO:teuthology.orchestra.run.smithi167.stdout:[mon.c] 2022-11-27T16:14:11.628 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2022-11-27T16:14:11.629 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2022-11-27T16:14:11.629 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2022-11-27T16:14:11.629 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2022-11-27T16:14:11.629 INFO:teuthology.orchestra.run.smithi167.stdout:[mds] 2022-11-27T16:14:11.629 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds = 20 2022-11-27T16:14:11.630 INFO:teuthology.orchestra.run.smithi167.stdout: debug mds balancer = 20 2022-11-27T16:14:11.630 INFO:teuthology.orchestra.run.smithi167.stdout: debug ms = 1 2022-11-27T16:14:11.630 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug frag = True 2022-11-27T16:14:11.630 INFO:teuthology.orchestra.run.smithi167.stdout: mds debug scatterstat = True 2022-11-27T16:14:11.631 INFO:teuthology.orchestra.run.smithi167.stdout: mds op complaint time = 180 2022-11-27T16:14:11.631 INFO:teuthology.orchestra.run.smithi167.stdout: mds verify scatter = True 2022-11-27T16:14:11.631 INFO:teuthology.orchestra.run.smithi167.stdout: osd op complaint time = 180 2022-11-27T16:14:11.631 INFO:teuthology.orchestra.run.smithi167.stdout: rados mon op timeout = 900 2022-11-27T16:14:11.632 INFO:teuthology.orchestra.run.smithi167.stdout: rados osd op timeout = 900 2022-11-27T16:14:11.632 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T16:14:11.633 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T16:14:11.691 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T16:14:11.692 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:11.692 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T16:14:11.710 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:11.710 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T16:14:11.767 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi164.front.sepia.ceph.com 2022-11-27T16:14:11.768 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:11.768 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:14:11.769 DEBUG:teuthology.orchestra.run.smithi164:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:14:11.854 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:11.854 DEBUG:teuthology.orchestra.run.smithi164:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:11.911 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi167.front.sepia.ceph.com 2022-11-27T16:14:11.912 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:11.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:14:11.912 DEBUG:teuthology.orchestra.run.smithi167:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:14:11.952 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:11.953 DEBUG:teuthology.orchestra.run.smithi167:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:12.010 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T16:14:12.011 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T16:14:12.011 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:14:12.065 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:14:12.068 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:14:12.154 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:14:12.157 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T16:14:12.157 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:14:12.211 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:14:12.214 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T16:14:12.240 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:14:12.334 INFO:teuthology.orchestra.run.smithi164.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:14:12.337 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T16:14:12.364 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:14:12.419 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:14:12.422 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T16:14:12.449 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:14:12.543 INFO:teuthology.orchestra.run.smithi167.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:14:12.546 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-11-27T16:14:12.573 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T16:14:12.573 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T16:14:12.617 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T16:14:12.631 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-11-27T16:14:12.714 INFO:teuthology.orchestra.run.smithi164.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-11-27T16:14:12.728 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T16:14:12.772 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T16:14:12.786 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-11-27T16:14:12.869 INFO:teuthology.orchestra.run.smithi167.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-11-27T16:14:12.881 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T16:14:12.882 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi164.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi167.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:14:12.882 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T16:14:12.911 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:12.911 INFO:tasks.ceph:role: osd.0 2022-11-27T16:14:12.912 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com 2022-11-27T16:14:12.912 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:14:12.986 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:12.987 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:12.987 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:12.987 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:12.988 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:12.988 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:12.988 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:12.989 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:12.990 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:12.990 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:13.419 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2022-11-27T16:14:13.421 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2022-11-27T16:14:13.422 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T16:14:13.587 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T16:14:13.625 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T16:14:13.695 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:13.695 INFO:tasks.ceph:role: osd.1 2022-11-27T16:14:13.696 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com 2022-11-27T16:14:13.696 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:14:13.771 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:13.772 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:13.772 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:13.772 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:13.773 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:13.773 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:13.773 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:13.773 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:13.774 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:13.774 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:14.188 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2022-11-27T16:14:14.191 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2022-11-27T16:14:14.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T16:14:14.306 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T16:14:14.341 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T16:14:14.410 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:14.410 INFO:tasks.ceph:role: osd.2 2022-11-27T16:14:14.410 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com 2022-11-27T16:14:14.411 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:14:14.487 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:14.487 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:14.488 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:14.488 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:14.488 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:14.488 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:14.488 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:14.489 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:14.489 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:14.489 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:14.943 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2022-11-27T16:14:14.945 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2022-11-27T16:14:14.946 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T16:14:15.081 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T16:14:15.117 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T16:14:15.190 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:15.191 INFO:tasks.ceph:role: osd.3 2022-11-27T16:14:15.191 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com 2022-11-27T16:14:15.191 DEBUG:teuthology.orchestra.run.smithi164:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:14:15.268 INFO:teuthology.orchestra.run.smithi164.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:15.269 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:15.269 INFO:teuthology.orchestra.run.smithi164.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:15.269 INFO:teuthology.orchestra.run.smithi164.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:15.270 INFO:teuthology.orchestra.run.smithi164.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:15.271 INFO:teuthology.orchestra.run.smithi164.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:15.271 INFO:teuthology.orchestra.run.smithi164.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:15.272 INFO:teuthology.orchestra.run.smithi164.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:15.272 INFO:teuthology.orchestra.run.smithi164.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:15.273 INFO:teuthology.orchestra.run.smithi164.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:15.708 INFO:teuthology.orchestra.run.smithi164.stdout:Discarding blocks...Done. 2022-11-27T16:14:15.711 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi164.front.sepia.ceph.com -o noatime 2022-11-27T16:14:15.711 DEBUG:teuthology.orchestra.run.smithi164:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T16:14:15.890 DEBUG:teuthology.orchestra.run.smithi164:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T16:14:15.926 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:16.019 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:16.016+0000 7f05753035c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T16:14:16.020 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:16.017+0000 7f05753035c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T16:14:16.020 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:16.017+0000 7f05753035c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:14:16.020 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:16.017+0000 7f05753035c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:14:16.021 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:16.017+0000 7f05753035c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2022-11-27T16:14:16.021 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:16.017+0000 7f05753035c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2022-11-27T16:14:18.891 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T16:14:18.923 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:19.016 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:19.014+0000 7f327d1795c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T16:14:19.016 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:19.014+0000 7f327d1795c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T16:14:19.017 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:19.014+0000 7f327d1795c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:14:19.017 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:19.014+0000 7f327d1795c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:14:19.017 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:19.014+0000 7f327d1795c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2022-11-27T16:14:19.018 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:19.014+0000 7f327d1795c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2022-11-27T16:14:21.875 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T16:14:21.912 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:22.015 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:22.011+0000 7fcd3eab85c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T16:14:22.016 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:22.012+0000 7fcd3eab85c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T16:14:22.016 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:22.012+0000 7fcd3eab85c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:14:22.017 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:22.012+0000 7fcd3eab85c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:14:22.017 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:22.012+0000 7fcd3eab85c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2022-11-27T16:14:22.018 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:22.012+0000 7fcd3eab85c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2022-11-27T16:14:24.875 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T16:14:24.948 DEBUG:teuthology.orchestra.run.smithi164:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:25.042+0000 7f95f345e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T16:14:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:25.042+0000 7f95f345e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T16:14:25.044 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:25.042+0000 7f95f345e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:14:25.045 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:25.042+0000 7f95f345e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:14:25.045 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:25.042+0000 7f95f345e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2022-11-27T16:14:25.045 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:25.043+0000 7f95f345e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2022-11-27T16:14:27.907 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T16:14:27.946 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-11-27T16:14:27.981 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:27.981 INFO:tasks.ceph:role: osd.4 2022-11-27T16:14:27.982 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com 2022-11-27T16:14:27.982 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:14:28.059 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:28.060 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:28.060 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:28.060 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:28.061 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:28.061 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:28.061 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:28.061 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:28.061 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:28.062 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:28.500 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2022-11-27T16:14:28.501 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2022-11-27T16:14:28.502 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-11-27T16:14:28.692 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-11-27T16:14:28.731 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-11-27T16:14:28.803 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:28.804 INFO:tasks.ceph:role: osd.5 2022-11-27T16:14:28.804 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com 2022-11-27T16:14:28.805 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:14:28.881 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:28.881 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:28.882 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:28.882 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:28.883 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:28.883 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:28.884 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:28.884 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:28.884 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:29.288 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2022-11-27T16:14:29.290 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2022-11-27T16:14:29.291 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-11-27T16:14:29.404 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-11-27T16:14:29.438 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-11-27T16:14:29.507 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:29.508 INFO:tasks.ceph:role: osd.6 2022-11-27T16:14:29.509 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com 2022-11-27T16:14:29.509 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:14:29.585 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:29.586 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:29.586 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:29.586 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:29.587 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:29.587 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:29.587 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:29.587 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:29.588 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:29.588 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:30.033 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2022-11-27T16:14:30.035 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2022-11-27T16:14:30.035 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-11-27T16:14:30.190 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-11-27T16:14:30.227 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-11-27T16:14:30.297 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:14:30.298 INFO:tasks.ceph:role: osd.7 2022-11-27T16:14:30.298 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com 2022-11-27T16:14:30.299 DEBUG:teuthology.orchestra.run.smithi167:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:14:30.376 INFO:teuthology.orchestra.run.smithi167.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:14:30.376 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:14:30.376 INFO:teuthology.orchestra.run.smithi167.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:14:30.377 INFO:teuthology.orchestra.run.smithi167.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:14:30.377 INFO:teuthology.orchestra.run.smithi167.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:14:30.377 INFO:teuthology.orchestra.run.smithi167.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:14:30.377 INFO:teuthology.orchestra.run.smithi167.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:14:30.378 INFO:teuthology.orchestra.run.smithi167.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:14:30.378 INFO:teuthology.orchestra.run.smithi167.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:14:30.378 INFO:teuthology.orchestra.run.smithi167.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:14:30.810 INFO:teuthology.orchestra.run.smithi167.stdout:Discarding blocks...Done. 2022-11-27T16:14:30.812 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi167.front.sepia.ceph.com -o noatime 2022-11-27T16:14:30.813 DEBUG:teuthology.orchestra.run.smithi167:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-11-27T16:14:31.005 DEBUG:teuthology.orchestra.run.smithi167:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-11-27T16:14:31.042 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:31.131 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:31.129+0000 7f50bae9b5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-11-27T16:14:31.132 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:31.129+0000 7f50bae9b5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-11-27T16:14:31.132 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:31.129+0000 7f50bae9b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:14:31.132 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:31.129+0000 7f50bae9b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:14:31.132 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:31.129+0000 7f50bae9b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2022-11-27T16:14:31.133 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:31.130+0000 7f50bae9b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2022-11-27T16:14:33.991 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-11-27T16:14:34.023 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:34.114+0000 7fc5aed685c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-11-27T16:14:34.116 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:34.114+0000 7fc5aed685c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-11-27T16:14:34.117 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:34.114+0000 7fc5aed685c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:14:34.117 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:34.114+0000 7fc5aed685c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:14:34.117 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:34.114+0000 7fc5aed685c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2022-11-27T16:14:34.117 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:34.116+0000 7fc5aed685c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2022-11-27T16:14:36.978 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-11-27T16:14:37.011 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:37.101 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:37.099+0000 7f8f4d4a05c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-11-27T16:14:37.102 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:37.099+0000 7f8f4d4a05c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-11-27T16:14:37.102 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:37.099+0000 7f8f4d4a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:14:37.102 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:37.099+0000 7f8f4d4a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:14:37.102 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:37.099+0000 7f8f4d4a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2022-11-27T16:14:37.103 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:37.100+0000 7f8f4d4a05c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2022-11-27T16:14:39.430 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:14:39.434 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:14:39.962 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-11-27T16:14:39.993 DEBUG:teuthology.orchestra.run.smithi167:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:40.083 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:40.081+0000 7fc9ed50e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-11-27T16:14:40.083 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:40.081+0000 7fc9ed50e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-11-27T16:14:40.084 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:40.081+0000 7fc9ed50e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:14:40.084 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:40.081+0000 7fc9ed50e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:14:40.084 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:40.081+0000 7fc9ed50e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2022-11-27T16:14:40.084 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:40.082+0000 7fc9ed50e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2022-11-27T16:14:42.940 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-11-27T16:14:42.973 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T16:14:42.973 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:42.974 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T16:14:43.007 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.007 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T16:14:43.078 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.078 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T16:14:43.148 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.148 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T16:14:43.216 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.217 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T16:14:43.284 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.285 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T16:14:43.351 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.351 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T16:14:43.418 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.418 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T16:14:43.450 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.451 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T16:14:43.521 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.522 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-11-27T16:14:43.590 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.591 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-11-27T16:14:43.658 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.659 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-11-27T16:14:43.726 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.727 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-11-27T16:14:43.794 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.795 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-11-27T16:14:43.861 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.862 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T16:14:43.881 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:43.882 DEBUG:teuthology.orchestra.run.smithi164:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-11-27T16:14:43.941 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.941 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T16:14:43.961 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:43.963 DEBUG:teuthology.orchestra.run.smithi167:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-11-27T16:14:44.021 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T16:14:44.022 DEBUG:teuthology.orchestra.run.smithi164:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:14:44.026 DEBUG:teuthology.orchestra.run.smithi167:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:14:44.065 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.y] 2022-11-27T16:14:44.065 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjwujfAxAAoNySHGGj04QKkz+FA50VcQ== 2022-11-27T16:14:44.065 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.a] 2022-11-27T16:14:44.066 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINj/leYDBAAX9fFcdW0iy8/jNQr6PuaZg== 2022-11-27T16:14:44.066 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.c] 2022-11-27T16:14:44.066 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjTDrpExAAkIRLuOQJYXjtGAjd7wWZMg== 2022-11-27T16:14:44.066 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.0] 2022-11-27T16:14:44.066 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDYjINj4cMLARAAkU/ELRmCjJMkuSmCqJX5+w== 2022-11-27T16:14:44.067 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.1] 2022-11-27T16:14:44.067 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDbjINjVGThABAA3OLK5iJ7Ws/9htZ4oQOhjg== 2022-11-27T16:14:44.067 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.2] 2022-11-27T16:14:44.067 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDejINjIoG+ABAA/lGUD0ioxYz7bRBDyK+lDw== 2022-11-27T16:14:44.068 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.3] 2022-11-27T16:14:44.068 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDhjINjRU2OAhAA4Gbd4w8JgHYYOBZYhsKNIg== 2022-11-27T16:14:44.068 INFO:teuthology.orchestra.run.smithi164.stdout:[mgr.x] 2022-11-27T16:14:44.068 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjRikoCRAANoiSq7Z1AkrpguFcvBJYDQ== 2022-11-27T16:14:44.068 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.b] 2022-11-27T16:14:44.069 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjABHsGBAA1PE/on3BpBynkoj24GriKw== 2022-11-27T16:14:44.069 INFO:teuthology.orchestra.run.smithi164.stdout:[mds.d] 2022-11-27T16:14:44.070 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjZdNaIBAAM/llOc8cT6q++Zy8NQM+Gw== 2022-11-27T16:14:44.070 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.4] 2022-11-27T16:14:44.070 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDnjINjc2HHBxAAgVkAq+40UwgZuU/Ok098Hg== 2022-11-27T16:14:44.070 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.5] 2022-11-27T16:14:44.071 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDqjINjT/3fBhAA1r/XQYqtUwp+Pii36loAIw== 2022-11-27T16:14:44.071 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.6] 2022-11-27T16:14:44.072 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDtjINjgvT6BRAAxBg1oBGdB8WKewH1BZTwZA== 2022-11-27T16:14:44.072 INFO:teuthology.orchestra.run.smithi164.stdout:[osd.7] 2022-11-27T16:14:44.073 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDwjINjXxXnBBAAVwvqn/XICGy71YdwFMpW1g== 2022-11-27T16:14:44.073 INFO:teuthology.orchestra.run.smithi164.stdout:[client.0] 2022-11-27T16:14:44.074 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjXtnEJBAAI/OQtufMUw2YGGr4iK2WnA== 2022-11-27T16:14:44.074 INFO:teuthology.orchestra.run.smithi164.stdout:[client.2] 2022-11-27T16:14:44.074 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjn9qMKhAA4/3Y3XI27oNaUpZuSeRxWw== 2022-11-27T16:14:44.075 INFO:teuthology.orchestra.run.smithi164.stdout:[client.1] 2022-11-27T16:14:44.075 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINj0xoALhAAxBddwnuFXNBBegYh4e/Eyw== 2022-11-27T16:14:44.075 INFO:teuthology.orchestra.run.smithi164.stdout:[client.3] 2022-11-27T16:14:44.076 INFO:teuthology.orchestra.run.smithi164.stdout: key = AQDUjINjv9a9MxAAwTvisb3gp0Zp8YFVJZDymQ== 2022-11-27T16:14:44.086 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.y] 2022-11-27T16:14:44.087 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjwujfAxAAoNySHGGj04QKkz+FA50VcQ== 2022-11-27T16:14:44.087 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.a] 2022-11-27T16:14:44.087 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINj/leYDBAAX9fFcdW0iy8/jNQr6PuaZg== 2022-11-27T16:14:44.088 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.c] 2022-11-27T16:14:44.088 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjTDrpExAAkIRLuOQJYXjtGAjd7wWZMg== 2022-11-27T16:14:44.088 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.0] 2022-11-27T16:14:44.088 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDYjINj4cMLARAAkU/ELRmCjJMkuSmCqJX5+w== 2022-11-27T16:14:44.088 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.1] 2022-11-27T16:14:44.089 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDbjINjVGThABAA3OLK5iJ7Ws/9htZ4oQOhjg== 2022-11-27T16:14:44.089 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.2] 2022-11-27T16:14:44.089 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDejINjIoG+ABAA/lGUD0ioxYz7bRBDyK+lDw== 2022-11-27T16:14:44.089 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.3] 2022-11-27T16:14:44.090 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDhjINjRU2OAhAA4Gbd4w8JgHYYOBZYhsKNIg== 2022-11-27T16:14:44.090 INFO:teuthology.orchestra.run.smithi167.stdout:[mgr.x] 2022-11-27T16:14:44.090 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjRikoCRAANoiSq7Z1AkrpguFcvBJYDQ== 2022-11-27T16:14:44.090 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.b] 2022-11-27T16:14:44.090 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjABHsGBAA1PE/on3BpBynkoj24GriKw== 2022-11-27T16:14:44.091 INFO:teuthology.orchestra.run.smithi167.stdout:[mds.d] 2022-11-27T16:14:44.091 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjZdNaIBAAM/llOc8cT6q++Zy8NQM+Gw== 2022-11-27T16:14:44.091 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.4] 2022-11-27T16:14:44.091 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDnjINjc2HHBxAAgVkAq+40UwgZuU/Ok098Hg== 2022-11-27T16:14:44.092 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.5] 2022-11-27T16:14:44.092 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDqjINjT/3fBhAA1r/XQYqtUwp+Pii36loAIw== 2022-11-27T16:14:44.092 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.6] 2022-11-27T16:14:44.092 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDtjINjgvT6BRAAxBg1oBGdB8WKewH1BZTwZA== 2022-11-27T16:14:44.092 INFO:teuthology.orchestra.run.smithi167.stdout:[osd.7] 2022-11-27T16:14:44.093 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDwjINjXxXnBBAAVwvqn/XICGy71YdwFMpW1g== 2022-11-27T16:14:44.093 INFO:teuthology.orchestra.run.smithi167.stdout:[client.0] 2022-11-27T16:14:44.093 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjXtnEJBAAI/OQtufMUw2YGGr4iK2WnA== 2022-11-27T16:14:44.093 INFO:teuthology.orchestra.run.smithi167.stdout:[client.2] 2022-11-27T16:14:44.094 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjn9qMKhAA4/3Y3XI27oNaUpZuSeRxWw== 2022-11-27T16:14:44.094 INFO:teuthology.orchestra.run.smithi167.stdout:[client.1] 2022-11-27T16:14:44.094 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINj0xoALhAAxBddwnuFXNBBegYh4e/Eyw== 2022-11-27T16:14:44.094 INFO:teuthology.orchestra.run.smithi167.stdout:[client.3] 2022-11-27T16:14:44.094 INFO:teuthology.orchestra.run.smithi167.stdout: key = AQDUjINjv9a9MxAAwTvisb3gp0Zp8YFVJZDymQ== 2022-11-27T16:14:44.096 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:14:44.120 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:14:44.191 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.195 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.251 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.253 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.309 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.312 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.366 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.370 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.424 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.427 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.481 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.483 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.536 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:14:44.538 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:14:44.591 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.594 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.651 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.654 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:14:44.709 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.712 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.764 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.767 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.821 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.824 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.878 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.881 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:14:44.935 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:44.937 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:44.994 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:44.997 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:45.053 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:45.055 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:45.111 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:45.114 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:14:45.170 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T16:14:45.170 DEBUG:teuthology.orchestra.run.smithi164:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T16:14:45.197 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:14:45.294 DEBUG:teuthology.orchestra.run.smithi164:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T16:14:45.321 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T16:14:45.348 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:14:45.446 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T16:14:45.472 DEBUG:teuthology.orchestra.run.smithi167:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T16:14:45.539 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:14:45.636 DEBUG:teuthology.orchestra.run.smithi167:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T16:14:45.663 DEBUG:teuthology.orchestra.run.smithi164:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:45.666 DEBUG:teuthology.orchestra.run.smithi167:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:14:45.720 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T16:14:45.721 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T16:14:45.721 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T16:14:45.724 INFO:tasks.ceph.mon.a:Started 2022-11-27T16:14:45.724 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T16:14:45.724 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T16:14:45.763 INFO:tasks.ceph.mon.b:Started 2022-11-27T16:14:45.763 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T16:14:45.764 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T16:14:45.767 INFO:tasks.ceph.mon.c:Started 2022-11-27T16:14:45.767 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T16:14:45.768 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T16:14:45.769 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T16:14:45.772 INFO:tasks.ceph.mgr.y:Started 2022-11-27T16:14:45.772 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T16:14:45.773 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T16:14:45.776 INFO:tasks.ceph.mgr.x:Started 2022-11-27T16:14:45.776 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T16:14:45.777 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T16:14:45.902 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.900+0000 7f5008de3700 1 Processor -- start 2022-11-27T16:14:45.903 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.900+0000 7f5008de3700 1 -- start start 2022-11-27T16:14:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5008de3700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 0x7f50040ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5008de3700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 0x7f500415bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5008de3700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50040ccb60 con 0x7f500406c570 2022-11-27T16:14:45.905 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5008de3700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5004070920 con 0x7f5004159090 2022-11-27T16:14:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5008de3700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5004070c00 con 0x7f5004159460 2022-11-27T16:14:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f500259c700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 0x7f50040ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f500259c700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 0x7f50040ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36348/0 (socket says 172.21.15.164:36348) 2022-11-27T16:14:45.906 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f500259c700 1 -- 172.21.15.164:0/1547791943 learned_addr learned my addr 172.21.15.164:0/1547791943 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:14:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f500259c700 1 -- 172.21.15.164:0/1547791943 >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 msgr2=0x7f50040ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2022-11-27T16:14:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f500259c700 1 -- 172.21.15.164:0/1547791943 >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 msgr2=0x7f50040ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:14:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f500259c700 1 --2- 172.21.15.164:0/1547791943 >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 0x7f50040ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:14:45.907 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f500259c700 1 --2- 172.21.15.164:0/1547791943 >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 0x7f50040ceb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:14:45.908 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5001d9b700 1 --2- 172.21.15.164:0/1547791943 >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 0x7f500415bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:45.908 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5001d9b700 1 -- 172.21.15.164:0/1547791943 >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 msgr2=0x7f500415bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:14:45.908 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5001d9b700 1 -- 172.21.15.164:0/1547791943 >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 msgr2=0x7f500415bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:14:45.908 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5001d9b700 1 --2- 172.21.15.164:0/1547791943 >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 0x7f500415bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:14:45.908 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:45.901+0000 7f5001d9b700 1 --2- 172.21.15.164:0/1547791943 >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 0x7f500415bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:14:46.097 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.087+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151203922 0 0) 0x7f50040ccb60 con 0x7f500406c570 2022-11-27T16:14:46.098 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.087+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe80035b0 con 0x7f500406c570 2022-11-27T16:14:46.098 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 <== mon.1 v1:172.21.15.167:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363998730 0 0) 0x7f4fe80035b0 con 0x7f500406c570 2022-11-27T16:14:46.098 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 msgr2=0x7f500415bde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:14:46.099 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 --2- 172.21.15.164:0/1547791943 >> v2:172.21.15.167:3300/0 conn(0x7f5004159460 0x7f500415bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.099 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 msgr2=0x7f50040ceb40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:14:46.099 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 --2- 172.21.15.164:0/1547791943 >> v2:172.21.15.164:3300/0 conn(0x7f5004159090 0x7f50040ceb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.099 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 --> v1:172.21.15.167:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004070ee0 con 0x7f500406c570 2022-11-27T16:14:46.099 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 <== mon.1 v1:172.21.15.167:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 699751603 0 0) 0x7f4ffc0019f0 con 0x7f500406c570 2022-11-27T16:14:46.100 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 >> v1:172.21.15.167:6789/0 conn(0x7f500406c570 legacy=0x7f500415f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.100 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.100 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fe8004850 0x7f4fe8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.100 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe8005200 0x7f4fe800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004070920 con 0x7f4fe8005200 2022-11-27T16:14:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004070c00 con 0x7f4fe8003a00 2022-11-27T16:14:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe800c0e0 con 0x7f4fe8004850 2022-11-27T16:14:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500259c700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.101 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f5002d9d700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe8005200 0x7f4fe800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.102 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f5001d9b700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fe8004850 0x7f4fe8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.102 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500259c700 1 -- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fe8004850 msgr2=0x7f4fe8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:46.102 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500259c700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f4fe8004850 0x7f4fe8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.102 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500259c700 1 -- 172.21.15.164:0/1547791943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe8005200 msgr2=0x7f4fe800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:46.103 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500259c700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe8005200 0x7f4fe800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.103 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f500259c700 1 -- 172.21.15.164:0/1547791943 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80035b0 con 0x7f4fe8003a00 2022-11-27T16:14:46.103 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f5002d9d700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f4fe8005200 0x7f4fe800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:46.103 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.088+0000 7f5002d9d700 1 -- 172.21.15.164:0/1547791943 reap_dead start 2022-11-27T16:14:46.103 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.089+0000 7f500259c700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009910 tx=0x7f4fe8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d414b729e7bada2c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:46.104 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.089+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff400c070 con 0x7f4fe8003a00 2022-11-27T16:14:46.104 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.089+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4004020 con 0x7f4fe8003a00 2022-11-27T16:14:46.104 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.089+0000 7f500159a700 1 -- 172.21.15.164:0/1547791943 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40043d0 con 0x7f4fe8003a00 2022-11-27T16:14:46.104 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.089+0000 7f5008de3700 1 -- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 msgr2=0x7f4fe8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.105 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.089+0000 7f5008de3700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009910 tx=0x7f4fe8005880 comp rx=0 tx=0).stop 2022-11-27T16:14:46.105 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.090+0000 7f5008de3700 1 -- 172.21.15.164:0/1547791943 shutdown_connections 2022-11-27T16:14:46.105 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.090+0000 7f5008de3700 1 --2- 172.21.15.164:0/1547791943 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f4fe8003e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.105 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.090+0000 7f5008de3700 1 -- 172.21.15.164:0/1547791943 >> 172.21.15.164:0/1547791943 conn(0x7f50040cbb40 msgr2=0x7f50040ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:46.105 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.090+0000 7f5008de3700 1 -- 172.21.15.164:0/1547791943 shutdown_connections 2022-11-27T16:14:46.106 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.090+0000 7f5008de3700 1 -- 172.21.15.164:0/1547791943 wait complete. 2022-11-27T16:14:46.106 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.092+0000 7f5008de3700 1 Processor -- start 2022-11-27T16:14:46.106 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.092+0000 7f5008de3700 1 -- start start 2022-11-27T16:14:46.106 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5008de3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f5004150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.107 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5008de3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50040b2b40 0x7f50040b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.107 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5008de3700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50040c43e0 0x7f50040c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.107 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5008de3700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c4d90 con 0x7f50040c43e0 2022-11-27T16:14:46.107 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5008de3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c20c0 con 0x7f4fe8003a00 2022-11-27T16:14:46.107 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5008de3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50040b3bc0 con 0x7f50040b2b40 2022-11-27T16:14:46.108 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f5004150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.108 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5002d9d700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50040c43e0 0x7f50040c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.108 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f5004150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.164:37618/0 (socket says 172.21.15.164:37618) 2022-11-27T16:14:46.108 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 -- 172.21.15.164:0/2202859061 learned_addr learned my addr 172.21.15.164:0/2202859061 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:14:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f5001d9b700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50040b2b40 0x7f50040b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 -- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50040b2b40 msgr2=0x7f50040b2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50040b2b40 0x7f50040b2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 -- 172.21.15.164:0/2202859061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50040c43e0 msgr2=0x7f50040c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.109 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50040c43e0 0x7f50040c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.093+0000 7f500259c700 1 -- 172.21.15.164:0/2202859061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50040b3600 con 0x7f4fe8003a00 2022-11-27T16:14:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.094+0000 7f500259c700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f5004150bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4016040 tx=0x7f4ff40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.094+0000 7f4ff2ffd700 1 -- 172.21.15.164:0/2202859061 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40045e0 con 0x7f4fe8003a00 2022-11-27T16:14:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.094+0000 7f4ff2ffd700 1 -- 172.21.15.164:0/2202859061 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4003d90 con 0x7f4fe8003a00 2022-11-27T16:14:46.110 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.094+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50040b38e0 con 0x7f4fe8003a00 2022-11-27T16:14:46.111 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.094+0000 7f4ff2ffd700 1 -- 172.21.15.164:0/2202859061 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4026660 con 0x7f4fe8003a00 2022-11-27T16:14:46.111 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.094+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5004151130 con 0x7f4fe8003a00 2022-11-27T16:14:46.111 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.094+0000 7f4ff2ffd700 1 -- 172.21.15.164:0/2202859061 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4ff402c020 con 0x7f4fe8003a00 2022-11-27T16:14:46.111 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.095+0000 7f4ff2ffd700 1 -- 172.21.15.164:0/2202859061 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4ff401d840 con 0x7f4fe8003a00 2022-11-27T16:14:46.112 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.095+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50040c1f20 con 0x7f4fe8003a00 2022-11-27T16:14:46.112 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.099+0000 7f4ff2ffd700 1 -- 172.21.15.164:0/2202859061 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f50040c1f20 con 0x7f4fe8003a00 2022-11-27T16:14:46.177 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.174+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5004073080 con 0x7f4fe8003a00 2022-11-27T16:14:46.184 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.182+0000 7f4ff2ffd700 1 -- 172.21.15.164:0/2202859061 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4ff4017030 con 0x7f4fe8003a00 2022-11-27T16:14:46.187 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 msgr2=0x7f5004150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.188 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f5004150bf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4016040 tx=0x7f4ff40047c0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.189 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 shutdown_connections 2022-11-27T16:14:46.189 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f50040c43e0 0x7f50040c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.189 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f4fe8003a00 0x7f5004150bf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.190 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 --2- 172.21.15.164:0/2202859061 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f50040b2b40 0x7f50040b2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.190 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 >> 172.21.15.164:0/2202859061 conn(0x7f50040cbb40 msgr2=0x7f500415f690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:46.190 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 shutdown_connections 2022-11-27T16:14:46.190 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:46.183+0000 7f5008de3700 1 -- 172.21.15.164:0/2202859061 wait complete. 2022-11-27T16:14:46.191 INFO:teuthology.orchestra.run.smithi164.stderr:adjusted tunables profile to default 2022-11-27T16:14:46.198 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T16:14:46.198 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:46.199 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T16:14:46.206 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:46.204+0000 7ff5c49b9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:14:46.207 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:46.205+0000 7f30372cc200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:14:46.225 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:46.226 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T16:14:46.270 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:46.268+0000 7f30372cc200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:14:46.274 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:46.272+0000 7ff5c49b9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:14:46.291 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:46.292 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T16:14:46.343 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:46.341+0000 7f30372cc200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:14:46.352 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:46.350+0000 7ff5c49b9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:14:46.357 DEBUG:teuthology.orchestra.run.smithi164:> set -ex 2022-11-27T16:14:46.358 DEBUG:teuthology.orchestra.run.smithi164:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T16:14:46.420 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:46.418+0000 7f30372cc200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:14:46.424 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:46.424 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-11-27T16:14:46.434 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:46.432+0000 7ff5c49b9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:14:46.450 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:46.451 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-11-27T16:14:46.518 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:46.518 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-11-27T16:14:46.569 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:46.568+0000 7f30372cc200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:14:46.574 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:46.572+0000 7ff5c49b9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:14:46.586 DEBUG:teuthology.orchestra.run.smithi167:> set -ex 2022-11-27T16:14:46.586 DEBUG:teuthology.orchestra.run.smithi167:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-11-27T16:14:46.653 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new fc87cc2c-4aba-41f8-9b94-8843abd34341 0 2022-11-27T16:14:46.818 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.813+0000 7fee9b083700 1 Processor -- start 2022-11-27T16:14:46.818 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.813+0000 7fee9b083700 1 -- start start 2022-11-27T16:14:46.819 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee9b083700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee9406b2f0 0x7fee940ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.819 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee9b083700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee940b5f10 0x7fee940ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.819 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee9b083700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee940cc6a0 con 0x7fee9415b700 2022-11-27T16:14:46.819 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee9b083700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee940ccc60 con 0x7fee9406b2f0 2022-11-27T16:14:46.820 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee9b083700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee940cc980 con 0x7fee940b5f10 2022-11-27T16:14:46.820 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee940b5f10 0x7fee940ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.820 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee93fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee9406b2f0 0x7fee940ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.820 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee940b5f10 0x7fee940ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57852/0 (socket says 172.21.15.167:57852) 2022-11-27T16:14:46.821 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 -- 172.21.15.167:0/1001896772 learned_addr learned my addr 172.21.15.167:0/1001896772 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:46.821 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee93fff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fee9406b2f0 0x7fee940ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52290/0 (socket says 172.21.15.167:52290) 2022-11-27T16:14:46.821 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 -- 172.21.15.167:0/1001896772 >> v2:172.21.15.167:3300/0 conn(0x7fee9406b2f0 msgr2=0x7fee940ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:46.821 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 --2- 172.21.15.167:0/1001896772 >> v2:172.21.15.167:3300/0 conn(0x7fee9406b2f0 0x7fee940ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.822 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 -- 172.21.15.167:0/1001896772 >> v1:172.21.15.164:6789/0 conn(0x7fee9415b700 legacy=0x7fee9406abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.822 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 -- 172.21.15.167:0/1001896772 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940d06c0 con 0x7fee940b5f10 2022-11-27T16:14:46.822 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee93fff700 1 --2- 172.21.15.167:0/1001896772 >> v2:172.21.15.167:3300/0 conn(0x7fee9406b2f0 0x7fee940ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:46.822 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee99620700 1 --2- 172.21.15.167:0/1001896772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee940b5f10 0x7fee940ce630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009e50 tx=0x7fee8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f668819eb832d3 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:46.822 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee937fe700 1 -- 172.21.15.167:0/1001896772 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00e070 con 0x7fee940b5f10 2022-11-27T16:14:46.823 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee937fe700 1 -- 172.21.15.167:0/1001896772 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c00b910 con 0x7fee940b5f10 2022-11-27T16:14:46.823 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.814+0000 7fee937fe700 1 -- 172.21.15.167:0/1001896772 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00bcc0 con 0x7fee940b5f10 2022-11-27T16:14:46.823 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 -- 172.21.15.167:0/1001896772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee940b5f10 msgr2=0x7fee940ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.823 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 --2- 172.21.15.167:0/1001896772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee940b5f10 0x7fee940ce630 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee8c009e50 tx=0x7fee8c00b330 comp rx=0 tx=0).stop 2022-11-27T16:14:46.824 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 -- 172.21.15.167:0/1001896772 shutdown_connections 2022-11-27T16:14:46.824 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 --2- 172.21.15.167:0/1001896772 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee940b5f10 0x7fee940ce630 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.824 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 --2- 172.21.15.167:0/1001896772 >> v2:172.21.15.167:3300/0 conn(0x7fee9406b2f0 0x7fee940ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.824 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 -- 172.21.15.167:0/1001896772 >> 172.21.15.167:0/1001896772 conn(0x7fee94070b00 msgr2=0x7fee94070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:46.825 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 -- 172.21.15.167:0/1001896772 shutdown_connections 2022-11-27T16:14:46.825 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.815+0000 7fee9b083700 1 -- 172.21.15.167:0/1001896772 wait complete. 2022-11-27T16:14:46.825 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.816+0000 7fee9b083700 1 Processor -- start 2022-11-27T16:14:46.825 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.816+0000 7fee9b083700 1 -- start start 2022-11-27T16:14:46.829 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee9b083700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee9406b2f0 0x7fee940ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.829 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee9b083700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee940b5f10 0x7fee941566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.830 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee9b083700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee9415b700 0x7fee9415ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:46.830 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee9b083700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee940c6240 con 0x7fee9415b700 2022-11-27T16:14:46.831 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee9b083700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9415ca50 con 0x7fee940b5f10 2022-11-27T16:14:46.831 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee9b083700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9415cd30 con 0x7fee9406b2f0 2022-11-27T16:14:46.832 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee93fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee940b5f10 0x7fee941566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.832 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee98e1f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee9406b2f0 0x7fee940ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.833 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee93fff700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee940b5f10 0x7fee941566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52294/0 (socket says 172.21.15.167:52294) 2022-11-27T16:14:46.834 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee93fff700 1 -- 172.21.15.167:0/3655045303 learned_addr learned my addr 172.21.15.167:0/3655045303 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:46.834 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee99620700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee9415b700 0x7fee9415ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:46.834 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee98e1f700 1 -- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee940b5f10 msgr2=0x7fee941566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.835 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee98e1f700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee940b5f10 0x7fee941566b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.835 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee98e1f700 1 -- 172.21.15.167:0/3655045303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee9415b700 msgr2=0x7fee9415ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.836 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee98e1f700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee9415b700 0x7fee9415ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.836 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee98e1f700 1 -- 172.21.15.167:0/3655045303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee800094a0 con 0x7fee9406b2f0 2022-11-27T16:14:46.837 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.817+0000 7fee93fff700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee940b5f10 0x7fee941566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:46.838 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee98e1f700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee9406b2f0 0x7fee940ca300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee88000f60 tx=0x7fee8800c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:46.838 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee917fa700 1 -- 172.21.15.167:0/3655045303 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee880052d0 con 0x7fee9406b2f0 2022-11-27T16:14:46.839 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee917fa700 1 -- 172.21.15.167:0/3655045303 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88005430 con 0x7fee9406b2f0 2022-11-27T16:14:46.839 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c009b30 con 0x7fee9406b2f0 2022-11-27T16:14:46.840 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee917fa700 1 -- 172.21.15.167:0/3655045303 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee880038c0 con 0x7fee9406b2f0 2022-11-27T16:14:46.840 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee940c5e60 con 0x7fee9406b2f0 2022-11-27T16:14:46.841 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee917fa700 1 -- 172.21.15.167:0/3655045303 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fee88019030 con 0x7fee9406b2f0 2022-11-27T16:14:46.841 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.818+0000 7fee917fa700 1 -- 172.21.15.167:0/3655045303 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7fee8801cd10 con 0x7fee9406b2f0 2022-11-27T16:14:46.842 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.819+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee940c6380 con 0x7fee9406b2f0 2022-11-27T16:14:46.842 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.821+0000 7fee917fa700 1 -- 172.21.15.167:0/3655045303 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fee88017070 con 0x7fee9406b2f0 2022-11-27T16:14:46.901 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:46.899+0000 7ff5c49b9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:14:46.905 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.903+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fc87cc2c-4aba-41f8-9b94-8843abd34341", "id": 0} v 0) v1 -- 0x7fee94073080 con 0x7fee9406b2f0 2022-11-27T16:14:46.910 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.909+0000 7fee917fa700 1 -- 172.21.15.167:0/3655045303 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fc87cc2c-4aba-41f8-9b94-8843abd34341", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fee8801c440 con 0x7fee9406b2f0 2022-11-27T16:14:46.911 INFO:teuthology.orchestra.run.smithi167.stdout:0 2022-11-27T16:14:46.912 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee9406b2f0 msgr2=0x7fee940ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:46.913 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee9406b2f0 0x7fee940ca300 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee88000f60 tx=0x7fee8800c6e0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.913 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 shutdown_connections 2022-11-27T16:14:46.913 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fee9415b700 0x7fee9415ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.914 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fee9406b2f0 0x7fee940ca300 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.914 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 --2- 172.21.15.167:0/3655045303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fee940b5f10 0x7fee941566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:46.914 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 >> 172.21.15.167:0/3655045303 conn(0x7fee94070b00 msgr2=0x7fee940d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:46.914 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 shutdown_connections 2022-11-27T16:14:46.914 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:46.910+0000 7fee9b083700 1 -- 172.21.15.167:0/3655045303 wait complete. 2022-11-27T16:14:46.925 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:46.918+0000 7f30372cc200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:14:46.926 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 2288745b-11fa-433b-bf88-dbfd0693401d 1 2022-11-27T16:14:46.975 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:46.973+0000 7ff5c49b9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:14:46.998 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:46.996+0000 7f30372cc200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:14:47.049 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.047+0000 7f0405acc700 1 Processor -- start 2022-11-27T16:14:47.050 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.047+0000 7f0405acc700 1 -- start start 2022-11-27T16:14:47.053 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.048+0000 7f0405acc700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f04000ce080 0x7f0400166810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.053 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.048+0000 7f0405acc700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040015f480 con 0x7f04000cccd0 2022-11-27T16:14:47.053 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.048+0000 7f0405acc700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040015f6d0 con 0x7f040015cad0 2022-11-27T16:14:47.053 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.048+0000 7f0405acc700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040015bdd0 con 0x7f04000ce080 2022-11-27T16:14:47.054 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.048+0000 7f03feffd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f040015cad0 0x7f04000cd970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37606/0 (socket says 172.21.15.167:37606) 2022-11-27T16:14:47.054 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.048+0000 7f03feffd700 1 -- 172.21.15.167:0/804135234 learned_addr learned my addr 172.21.15.167:0/804135234 (peer_addr_for_me v1:172.21.15.167:0/0) 2022-11-27T16:14:47.054 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.048+0000 7f03fffff700 1 --2- 172.21.15.167:0/804135234 >> v2:172.21.15.164:3300/0 conn(0x7f04000ce080 0x7f0400166810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.054 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fe7fc700 1 -- 172.21.15.167:0/804135234 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312471071 0 0) 0x7f040015f6d0 con 0x7f040015cad0 2022-11-27T16:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fe7fc700 1 -- 172.21.15.167:0/804135234 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03e00035b0 con 0x7f040015cad0 2022-11-27T16:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fe7fc700 1 -- 172.21.15.167:0/804135234 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143726705 0 0) 0x7f040015f480 con 0x7f04000cccd0 2022-11-27T16:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fe7fc700 1 -- 172.21.15.167:0/804135234 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f040015f6d0 con 0x7f04000cccd0 2022-11-27T16:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fffff700 1 -- 172.21.15.167:0/804135234 >> v1:172.21.15.167:6789/0 conn(0x7f040015cad0 legacy=0x7f04000cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.055 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fffff700 1 -- 172.21.15.167:0/804135234 >> v1:172.21.15.164:6789/0 conn(0x7f04000cccd0 legacy=0x7f0400070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.056 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fffff700 1 -- 172.21.15.167:0/804135234 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040015f9b0 con 0x7f04000ce080 2022-11-27T16:14:47.056 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fffff700 1 --2- 172.21.15.167:0/804135234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f0400166810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009840 tx=0x7f03f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380c2c3eee29c400 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.056 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fe7fc700 1 -- 172.21.15.167:0/804135234 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8011830 con 0x7f04000ce080 2022-11-27T16:14:47.056 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fe7fc700 1 -- 172.21.15.167:0/804135234 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f8011990 con 0x7f04000ce080 2022-11-27T16:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.049+0000 7f03fe7fc700 1 -- 172.21.15.167:0/804135234 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f8011d20 con 0x7f04000ce080 2022-11-27T16:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.050+0000 7f0405acc700 1 -- 172.21.15.167:0/804135234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 msgr2=0x7f0400166810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.050+0000 7f0405acc700 1 --2- 172.21.15.167:0/804135234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f0400166810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009840 tx=0x7f03f8009b50 comp rx=0 tx=0).stop 2022-11-27T16:14:47.057 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.050+0000 7f0405acc700 1 -- 172.21.15.167:0/804135234 shutdown_connections 2022-11-27T16:14:47.058 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.050+0000 7f0405acc700 1 --2- 172.21.15.167:0/804135234 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f0400166810 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.058 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.050+0000 7f0405acc700 1 -- 172.21.15.167:0/804135234 >> 172.21.15.167:0/804135234 conn(0x7f040006dee0 msgr2=0x7f040006e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.058 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.050+0000 7f0405acc700 1 -- 172.21.15.167:0/804135234 shutdown_connections 2022-11-27T16:14:47.058 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.050+0000 7f0405acc700 1 -- 172.21.15.167:0/804135234 wait complete. 2022-11-27T16:14:47.062 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.051+0000 7f0405acc700 1 Processor -- start 2022-11-27T16:14:47.062 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.052+0000 7f0405acc700 1 -- start start 2022-11-27T16:14:47.062 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f0405acc700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04000cccd0 0x7f04000c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.063 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f0405acc700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f04000ab540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.063 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f0405acc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f040015cad0 0x7f040015a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.063 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f0405acc700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04000abd50 con 0x7f04000ce080 2022-11-27T16:14:47.063 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f0405acc700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04000ad0d0 con 0x7f04000cccd0 2022-11-27T16:14:47.064 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f0405acc700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04000aba80 con 0x7f040015cad0 2022-11-27T16:14:47.064 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03ff7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04000cccd0 0x7f04000c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.064 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03fffff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f040015cad0 0x7f040015a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.064 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03ff7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04000cccd0 0x7f04000c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52304/0 (socket says 172.21.15.167:52304) 2022-11-27T16:14:47.064 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03ff7fe700 1 -- 172.21.15.167:0/425189244 learned_addr learned my addr 172.21.15.167:0/425189244 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:47.065 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03feffd700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f04000ab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.065 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03fffff700 1 -- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04000cccd0 msgr2=0x7f04000c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.065 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03fffff700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04000cccd0 0x7f04000c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.066 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03fffff700 1 -- 172.21.15.167:0/425189244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 msgr2=0x7f04000ab540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.066 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03fffff700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f04000ab540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.066 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03fffff700 1 -- 172.21.15.167:0/425189244 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f80094a0 con 0x7f040015cad0 2022-11-27T16:14:47.066 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03ff7fe700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04000cccd0 0x7f04000c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:47.067 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03feffd700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f04000ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:47.067 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03fffff700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f040015cad0 0x7f040015a250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03f8016000 tx=0x7f03f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.067 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03f7fff700 1 -- 172.21.15.167:0/425189244 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f801a6e0 con 0x7f040015cad0 2022-11-27T16:14:47.067 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03f7fff700 1 -- 172.21.15.167:0/425189244 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f801a860 con 0x7f040015cad0 2022-11-27T16:14:47.068 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04000ac7a0 con 0x7f040015cad0 2022-11-27T16:14:47.068 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.053+0000 7f03f7fff700 1 -- 172.21.15.167:0/425189244 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f801ac10 con 0x7f040015cad0 2022-11-27T16:14:47.068 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.054+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04000acd60 con 0x7f040015cad0 2022-11-27T16:14:47.068 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.054+0000 7f03f7fff700 1 -- 172.21.15.167:0/425189244 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f03f802b030 con 0x7f040015cad0 2022-11-27T16:14:47.069 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.054+0000 7f03f7fff700 1 -- 172.21.15.167:0/425189244 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f03f802eac0 con 0x7f040015cad0 2022-11-27T16:14:47.069 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.055+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0400073080 con 0x7f040015cad0 2022-11-27T16:14:47.069 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.057+0000 7f03f7fff700 1 -- 172.21.15.167:0/425189244 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f03f8029070 con 0x7f040015cad0 2022-11-27T16:14:47.126 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:47.124+0000 7f30372cc200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:14:47.127 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:47.125+0000 7ff5c49b9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:14:47.141 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.139+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2288745b-11fa-433b-bf88-dbfd0693401d", "id": 1} v 0) v1 -- 0x7f04000ad210 con 0x7f040015cad0 2022-11-27T16:14:47.147 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.144+0000 7f03f7fff700 1 -- 172.21.15.167:0/425189244 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2288745b-11fa-433b-bf88-dbfd0693401d", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f03f8015020 con 0x7f040015cad0 2022-11-27T16:14:47.148 INFO:teuthology.orchestra.run.smithi167.stdout:1 2022-11-27T16:14:47.149 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.145+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f040015cad0 msgr2=0x7f040015a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.150 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.145+0000 7f0405acc700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f040015cad0 0x7f040015a250 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03f8016000 tx=0x7f03f8004750 comp rx=0 tx=0).stop 2022-11-27T16:14:47.150 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.145+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 shutdown_connections 2022-11-27T16:14:47.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.146+0000 7f0405acc700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f040015cad0 0x7f040015a250 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.146+0000 7f0405acc700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f04000cccd0 0x7f04000c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.146+0000 7f0405acc700 1 --2- 172.21.15.167:0/425189244 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f04000ce080 0x7f04000ab540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.146+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 >> 172.21.15.167:0/425189244 conn(0x7f040006dee0 msgr2=0x7f04000b2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.152 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.146+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 shutdown_connections 2022-11-27T16:14:47.152 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.146+0000 7f0405acc700 1 -- 172.21.15.167:0/425189244 wait complete. 2022-11-27T16:14:47.159 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 82ed3e90-5d84-4fc3-ac13-a6a146bc60f7 2 2022-11-27T16:14:47.208 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:47.206+0000 7ff5c49b9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:14:47.212 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:47.210+0000 7f30372cc200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:14:47.281 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.279+0000 7fe334019700 1 Processor -- start 2022-11-27T16:14:47.282 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.280+0000 7fe334019700 1 -- start start 2022-11-27T16:14:47.289 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.280+0000 7fe334019700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fe32c159460 0x7fe32c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.290 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.280+0000 7fe334019700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0ccb60 con 0x7fe32c06c570 2022-11-27T16:14:47.290 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.280+0000 7fe334019700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c070920 con 0x7fe32c159090 2022-11-27T16:14:47.290 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.280+0000 7fe334019700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c15c4f0 con 0x7fe32c159460 2022-11-27T16:14:47.291 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.280+0000 7fe331db5700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7fe32c159090 0x7fe32c15bde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37616/0 (socket says 172.21.15.167:37616) 2022-11-27T16:14:47.291 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.280+0000 7fe331db5700 1 -- 172.21.15.167:0/1891236350 learned_addr learned my addr 172.21.15.167:0/1891236350 (peer_addr_for_me v1:172.21.15.167:0/0) 2022-11-27T16:14:47.291 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe3315b4700 1 --2- 172.21.15.167:0/1891236350 >> v2:172.21.15.164:3300/0 conn(0x7fe32c159460 0x7fe32c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.291 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe330db3700 1 -- 172.21.15.167:0/1891236350 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102994661 0 0) 0x7fe32c070920 con 0x7fe32c159090 2022-11-27T16:14:47.292 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe330db3700 1 -- 172.21.15.167:0/1891236350 --> v1:172.21.15.167:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3140035b0 con 0x7fe32c159090 2022-11-27T16:14:47.292 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe330db3700 1 -- 172.21.15.167:0/1891236350 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689017428 0 0) 0x7fe32c0ccb60 con 0x7fe32c06c570 2022-11-27T16:14:47.292 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe330db3700 1 -- 172.21.15.167:0/1891236350 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c070920 con 0x7fe32c06c570 2022-11-27T16:14:47.292 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe3315b4700 1 -- 172.21.15.167:0/1891236350 >> v1:172.21.15.167:6789/0 conn(0x7fe32c159090 legacy=0x7fe32c15bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.293 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe3315b4700 1 -- 172.21.15.167:0/1891236350 >> v1:172.21.15.164:6789/0 conn(0x7fe32c06c570 legacy=0x7fe32c15f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.293 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe3315b4700 1 -- 172.21.15.167:0/1891236350 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c070c00 con 0x7fe32c159460 2022-11-27T16:14:47.293 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe3315b4700 1 --2- 172.21.15.167:0/1891236350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159460 0x7fe32c0ceb40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009840 tx=0x7fe31c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df21fb92e3988e7 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.293 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe330db3700 1 -- 172.21.15.167:0/1891236350 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c011830 con 0x7fe32c159460 2022-11-27T16:14:47.293 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe330db3700 1 -- 172.21.15.167:0/1891236350 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c011990 con 0x7fe32c159460 2022-11-27T16:14:47.294 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.281+0000 7fe330db3700 1 -- 172.21.15.167:0/1891236350 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c011d20 con 0x7fe32c159460 2022-11-27T16:14:47.294 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.282+0000 7fe334019700 1 -- 172.21.15.167:0/1891236350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159460 msgr2=0x7fe32c0ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.294 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.282+0000 7fe334019700 1 --2- 172.21.15.167:0/1891236350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159460 0x7fe32c0ceb40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009840 tx=0x7fe31c009b50 comp rx=0 tx=0).stop 2022-11-27T16:14:47.295 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.282+0000 7fe334019700 1 -- 172.21.15.167:0/1891236350 shutdown_connections 2022-11-27T16:14:47.295 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.282+0000 7fe334019700 1 --2- 172.21.15.167:0/1891236350 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159460 0x7fe32c0ceb40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.296 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.282+0000 7fe334019700 1 -- 172.21.15.167:0/1891236350 >> 172.21.15.167:0/1891236350 conn(0x7fe32c0cbb40 msgr2=0x7fe32c0ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.296 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.282+0000 7fe334019700 1 -- 172.21.15.167:0/1891236350 shutdown_connections 2022-11-27T16:14:47.296 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.282+0000 7fe334019700 1 -- 172.21.15.167:0/1891236350 wait complete. 2022-11-27T16:14:47.296 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.283+0000 7fe334019700 1 Processor -- start 2022-11-27T16:14:47.297 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.283+0000 7fe334019700 1 -- start start 2022-11-27T16:14:47.297 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe334019700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe32c06c570 0x7fe32c14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.297 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe334019700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159090 0x7fe32c0c1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.297 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe334019700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe32c159460 0x7fe32c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.298 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe334019700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c14cdd0 con 0x7fe32c159090 2022-11-27T16:14:47.298 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe334019700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0c92c0 con 0x7fe32c159460 2022-11-27T16:14:47.298 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe334019700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0c95a0 con 0x7fe32c06c570 2022-11-27T16:14:47.298 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe32c06c570 0x7fe32c14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.299 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe3325b6700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe32c159460 0x7fe32c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.299 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe32c06c570 0x7fe32c14c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57880/0 (socket says 172.21.15.167:57880) 2022-11-27T16:14:47.299 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 -- 172.21.15.167:0/2313909303 learned_addr learned my addr 172.21.15.167:0/2313909303 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:47.299 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe3315b4700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159090 0x7fe32c0c1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.300 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 -- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe32c159460 msgr2=0x7fe32c0c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.300 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe32c159460 0x7fe32c0c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.300 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 -- 172.21.15.167:0/2313909303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159090 msgr2=0x7fe32c0c1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.301 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159090 0x7fe32c0c1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.301 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 -- 172.21.15.167:0/2313909303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0094a0 con 0x7fe32c06c570 2022-11-27T16:14:47.301 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe3325b6700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe32c159460 0x7fe32c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:47.301 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.284+0000 7fe331db5700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe32c06c570 0x7fe32c14c890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c700 tx=0x7fe32800ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.302 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.285+0000 7fe31a7fc700 1 -- 172.21.15.167:0/2313909303 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328005d70 con 0x7fe32c06c570 2022-11-27T16:14:47.302 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.285+0000 7fe31a7fc700 1 -- 172.21.15.167:0/2313909303 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3280070d0 con 0x7fe32c06c570 2022-11-27T16:14:47.302 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.285+0000 7fe31a7fc700 1 -- 172.21.15.167:0/2313909303 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328007440 con 0x7fe32c06c570 2022-11-27T16:14:47.302 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.285+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c0c9880 con 0x7fe32c06c570 2022-11-27T16:14:47.303 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.285+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c0c2460 con 0x7fe32c06c570 2022-11-27T16:14:47.303 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.285+0000 7fe31a7fc700 1 -- 172.21.15.167:0/2313909303 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe328019030 con 0x7fe32c06c570 2022-11-27T16:14:47.303 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.285+0000 7fe31a7fc700 1 -- 172.21.15.167:0/2313909303 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fe32801b7d0 con 0x7fe32c06c570 2022-11-27T16:14:47.304 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.286+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c0c9da0 con 0x7fe32c06c570 2022-11-27T16:14:47.304 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.288+0000 7fe31a7fc700 1 -- 172.21.15.167:0/2313909303 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fe32c0c9da0 con 0x7fe32c06c570 2022-11-27T16:14:47.372 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.369+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "82ed3e90-5d84-4fc3-ac13-a6a146bc60f7", "id": 2} v 0) v1 -- 0x7fe32c073080 con 0x7fe32c06c570 2022-11-27T16:14:47.379 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.377+0000 7fe31a7fc700 1 -- 172.21.15.167:0/2313909303 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "82ed3e90-5d84-4fc3-ac13-a6a146bc60f7", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe328021060 con 0x7fe32c06c570 2022-11-27T16:14:47.380 INFO:teuthology.orchestra.run.smithi167.stdout:2 2022-11-27T16:14:47.381 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.378+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe32c06c570 msgr2=0x7fe32c14c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.382 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.378+0000 7fe334019700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe32c06c570 0x7fe32c14c890 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe32800c700 tx=0x7fe32800ca10 comp rx=0 tx=0).stop 2022-11-27T16:14:47.382 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.378+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 shutdown_connections 2022-11-27T16:14:47.382 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.378+0000 7fe334019700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fe32c159460 0x7fe32c0c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.382 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.379+0000 7fe334019700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fe32c06c570 0x7fe32c14c890 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.383 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.379+0000 7fe334019700 1 --2- 172.21.15.167:0/2313909303 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fe32c159090 0x7fe32c0c1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.383 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.379+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 >> 172.21.15.167:0/2313909303 conn(0x7fe32c0cbb40 msgr2=0x7fe32c0ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.383 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.379+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 shutdown_connections 2022-11-27T16:14:47.383 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.379+0000 7fe334019700 1 -- 172.21.15.167:0/2313909303 wait complete. 2022-11-27T16:14:47.393 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 864ca05d-4b9e-4111-8130-dd230c4a663c 3 2022-11-27T16:14:47.514 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.512+0000 7f0df3462700 1 Processor -- start 2022-11-27T16:14:47.515 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.512+0000 7f0df3462700 1 -- start start 2022-11-27T16:14:47.521 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df3462700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.521 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df3462700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0dec06b120 0x7f0dec0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df3462700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec0cc6a0 con 0x7f0dec0d1a20 2022-11-27T16:14:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df3462700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec0ccc60 con 0x7f0dec06b120 2022-11-27T16:14:47.522 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df3462700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec06b5a0 con 0x7f0dec15b700 2022-11-27T16:14:47.523 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df11fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.523 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df09fd700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f0dec0d1a20 0x7f0dec0b4f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.167:37628/0 (socket says 172.21.15.167:37628) 2022-11-27T16:14:47.524 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df09fd700 1 -- 172.21.15.167:0/1047613671 learned_addr learned my addr 172.21.15.167:0/1047613671 (peer_addr_for_me v1:172.21.15.167:0/0) 2022-11-27T16:14:47.524 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.513+0000 7f0df19ff700 1 --2- 172.21.15.167:0/1047613671 >> v2:172.21.15.167:3300/0 conn(0x7f0dec06b120 0x7f0dec0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.524 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0df19ff700 1 -- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 msgr2=0x7f0dec06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:47.524 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0df19ff700 1 --2- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.525 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0df19ff700 1 -- 172.21.15.167:0/1047613671 >> v1:172.21.15.167:6789/0 conn(0x7f0dec0d1a20 legacy=0x7f0dec0b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.525 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0debfff700 1 -- 172.21.15.167:0/1047613671 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504033759 0 0) 0x7f0dec0cc6a0 con 0x7f0dec0d1a20 2022-11-27T16:14:47.525 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0df19ff700 1 -- 172.21.15.167:0/1047613671 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0cc980 con 0x7f0dec06b120 2022-11-27T16:14:47.525 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0df11fe700 1 --2- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0df19ff700 1 --2- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec06b120 0x7f0dec0cf100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009840 tx=0x7f0de4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db04298a709a676 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0debfff700 1 -- 172.21.15.167:0/1047613671 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4011830 con 0x7f0dec06b120 2022-11-27T16:14:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0debfff700 1 -- 172.21.15.167:0/1047613671 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de4011990 con 0x7f0dec06b120 2022-11-27T16:14:47.526 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.514+0000 7f0debfff700 1 -- 172.21.15.167:0/1047613671 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4011d20 con 0x7f0dec06b120 2022-11-27T16:14:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 -- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec06b120 msgr2=0x7f0dec0cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 --2- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec06b120 0x7f0dec0cf100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0de4009840 tx=0x7f0de4009b50 comp rx=0 tx=0).stop 2022-11-27T16:14:47.527 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 -- 172.21.15.167:0/1047613671 shutdown_connections 2022-11-27T16:14:47.528 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 --2- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec06b120 0x7f0dec0cf100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.528 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 --2- 172.21.15.167:0/1047613671 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.528 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 -- 172.21.15.167:0/1047613671 >> 172.21.15.167:0/1047613671 conn(0x7f0dec070b00 msgr2=0x7f0dec070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 -- 172.21.15.167:0/1047613671 shutdown_connections 2022-11-27T16:14:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.515+0000 7f0df3462700 1 -- 172.21.15.167:0/1047613671 wait complete. 2022-11-27T16:14:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.516+0000 7f0df3462700 1 Processor -- start 2022-11-27T16:14:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.516+0000 7f0df3462700 1 -- start start 2022-11-27T16:14:47.529 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df3462700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dec06b120 0x7f0dec0c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.530 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df3462700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0d1a20 0x7f0dec1f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.530 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df3462700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.530 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df3462700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec0c5c80 con 0x7f0dec06b120 2022-11-27T16:14:47.530 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df3462700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec15d540 con 0x7f0dec0d1a20 2022-11-27T16:14:47.531 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df3462700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec0c5f20 con 0x7f0dec15b700 2022-11-27T16:14:47.531 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0d1a20 0x7f0dec1f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.531 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df19ff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.531 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0d1a20 0x7f0dec1f15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52330/0 (socket says 172.21.15.167:52330) 2022-11-27T16:14:47.532 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 -- 172.21.15.167:0/215216514 learned_addr learned my addr 172.21.15.167:0/215216514 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:47.532 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df11fe700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dec06b120 0x7f0dec0c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.532 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 -- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 msgr2=0x7f0dec15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.532 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.533 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 -- 172.21.15.167:0/215216514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dec06b120 msgr2=0x7f0dec0c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.533 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dec06b120 0x7f0dec0c6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.533 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 -- 172.21.15.167:0/215216514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de0009a70 con 0x7f0dec0d1a20 2022-11-27T16:14:47.533 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df19ff700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:47.534 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df09fd700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0d1a20 0x7f0dec1f15d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80011e0 tx=0x7f0dd800ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.534 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0de9ffb700 1 -- 172.21.15.167:0/215216514 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8004c80 con 0x7f0dec0d1a20 2022-11-27T16:14:47.534 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0de9ffb700 1 -- 172.21.15.167:0/215216514 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd8004e00 con 0x7f0dec0d1a20 2022-11-27T16:14:47.534 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de40094a0 con 0x7f0dec0d1a20 2022-11-27T16:14:47.535 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0de9ffb700 1 -- 172.21.15.167:0/215216514 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8007310 con 0x7f0dec0d1a20 2022-11-27T16:14:47.535 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.517+0000 7f0de9ffb700 1 -- 172.21.15.167:0/215216514 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f0dd801a020 con 0x7f0dec0d1a20 2022-11-27T16:14:47.535 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.518+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec0c64e0 con 0x7f0dec0d1a20 2022-11-27T16:14:47.535 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.518+0000 7f0de9ffb700 1 -- 172.21.15.167:0/215216514 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f0dd8007570 con 0x7f0dec0d1a20 2022-11-27T16:14:47.536 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.518+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec065ba0 con 0x7f0dec0d1a20 2022-11-27T16:14:47.537 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.520+0000 7f0de9ffb700 1 -- 172.21.15.167:0/215216514 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f0dd80133c0 con 0x7f0dec0d1a20 2022-11-27T16:14:47.606 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.604+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "864ca05d-4b9e-4111-8130-dd230c4a663c", "id": 3} v 0) v1 -- 0x7f0dec0cd430 con 0x7f0dec0d1a20 2022-11-27T16:14:47.614 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.612+0000 7f0de9ffb700 1 -- 172.21.15.167:0/215216514 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "864ca05d-4b9e-4111-8130-dd230c4a663c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0dd8013a50 con 0x7f0dec0d1a20 2022-11-27T16:14:47.614 INFO:teuthology.orchestra.run.smithi167.stdout:3 2022-11-27T16:14:47.616 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0d1a20 msgr2=0x7f0dec1f15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.616 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0d1a20 0x7f0dec1f15d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0dd80011e0 tx=0x7f0dd800ca30 comp rx=0 tx=0).stop 2022-11-27T16:14:47.616 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 shutdown_connections 2022-11-27T16:14:47.616 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0dec15b700 0x7f0dec15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.617 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0dec06b120 0x7f0dec0c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.617 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 --2- 172.21.15.167:0/215216514 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0dec0d1a20 0x7f0dec1f15d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.617 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 >> 172.21.15.167:0/215216514 conn(0x7f0dec070b00 msgr2=0x7f0dec15a600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.617 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 shutdown_connections 2022-11-27T16:14:47.617 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.613+0000 7f0df3462700 1 -- 172.21.15.167:0/215216514 wait complete. 2022-11-27T16:14:47.627 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 94560d98-a6fa-494f-8407-d432c650a7f2 4 2022-11-27T16:14:47.642 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:47.640+0000 7ff5c49b9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:14:47.657 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:47.655+0000 7f30372cc200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:14:47.748 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.746+0000 7f133fba7700 1 Processor -- start 2022-11-27T16:14:47.749 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.747+0000 7f133fba7700 1 -- start start 2022-11-27T16:14:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.747+0000 7f133fba7700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1338159560 0x7f13380ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.747+0000 7f133fba7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f133815dcf0 0x7f13380ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.755 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.747+0000 7f133fba7700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13380cade0 con 0x7f13380cebe0 2022-11-27T16:14:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.747+0000 7f133fba7700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13380cd970 con 0x7f1338159560 2022-11-27T16:14:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.747+0000 7f133fba7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13380cab00 con 0x7f133815dcf0 2022-11-27T16:14:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.747+0000 7f133d943700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1338159560 0x7f13380ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.756 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d943700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f1338159560 0x7f13380ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52334/0 (socket says 172.21.15.167:52334) 2022-11-27T16:14:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d142700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f133815dcf0 0x7f13380ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d943700 1 -- 172.21.15.167:0/4289536958 learned_addr learned my addr 172.21.15.167:0/4289536958 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d142700 1 -- 172.21.15.167:0/4289536958 >> v2:172.21.15.167:3300/0 conn(0x7f1338159560 msgr2=0x7f13380ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:47.757 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d142700 1 --2- 172.21.15.167:0/4289536958 >> v2:172.21.15.167:3300/0 conn(0x7f1338159560 0x7f13380ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d142700 1 -- 172.21.15.167:0/4289536958 >> v1:172.21.15.167:6789/0 conn(0x7f13380cebe0 legacy=0x7f13380b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d142700 1 -- 172.21.15.167:0/4289536958 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13380cdbc0 con 0x7f133815dcf0 2022-11-27T16:14:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d943700 1 --2- 172.21.15.167:0/4289536958 >> v2:172.21.15.167:3300/0 conn(0x7f1338159560 0x7f13380ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133d142700 1 --2- 172.21.15.167:0/4289536958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f133815dcf0 0x7f13380ce6a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1328009840 tx=0x7f1328009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3b16735791b9f99 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.758 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133c941700 1 -- 172.21.15.167:0/4289536958 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328011830 con 0x7f133815dcf0 2022-11-27T16:14:47.759 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133c941700 1 -- 172.21.15.167:0/4289536958 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1328011990 con 0x7f133815dcf0 2022-11-27T16:14:47.759 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.748+0000 7f133c941700 1 -- 172.21.15.167:0/4289536958 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328011d20 con 0x7f133815dcf0 2022-11-27T16:14:47.759 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 -- 172.21.15.167:0/4289536958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f133815dcf0 msgr2=0x7f13380ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.759 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 --2- 172.21.15.167:0/4289536958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f133815dcf0 0x7f13380ce6a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1328009840 tx=0x7f1328009b50 comp rx=0 tx=0).stop 2022-11-27T16:14:47.760 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 -- 172.21.15.167:0/4289536958 shutdown_connections 2022-11-27T16:14:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 --2- 172.21.15.167:0/4289536958 >> v2:172.21.15.167:3300/0 conn(0x7f1338159560 0x7f13380ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.761 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 --2- 172.21.15.167:0/4289536958 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f133815dcf0 0x7f13380ce6a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 -- 172.21.15.167:0/4289536958 >> 172.21.15.167:0/4289536958 conn(0x7f133806dee0 msgr2=0x7f133806e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 -- 172.21.15.167:0/4289536958 shutdown_connections 2022-11-27T16:14:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.749+0000 7f133fba7700 1 -- 172.21.15.167:0/4289536958 wait complete. 2022-11-27T16:14:47.762 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.750+0000 7f133fba7700 1 Processor -- start 2022-11-27T16:14:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.750+0000 7f133fba7700 1 -- start start 2022-11-27T16:14:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133fba7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13380cebe0 0x7f13380c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133fba7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338159560 0x7f133815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.763 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133fba7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133815dcf0 0x7f1338153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:47.764 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133fba7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380c7230 con 0x7f133815dcf0 2022-11-27T16:14:47.764 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133fba7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13380c7510 con 0x7f1338159560 2022-11-27T16:14:47.764 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133fba7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13380c77f0 con 0x7f13380cebe0 2022-11-27T16:14:47.765 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338159560 0x7f133815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.765 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338159560 0x7f133815bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52342/0 (socket says 172.21.15.167:52342) 2022-11-27T16:14:47.765 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 -- 172.21.15.167:0/2565537355 learned_addr learned my addr 172.21.15.167:0/2565537355 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:47.766 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d943700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13380cebe0 0x7f13380c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.766 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133e144700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133815dcf0 0x7f1338153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:47.767 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 -- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13380cebe0 msgr2=0x7f13380c64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.767 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13380cebe0 0x7f13380c64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.768 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 -- 172.21.15.167:0/2565537355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133815dcf0 msgr2=0x7f1338153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.768 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133815dcf0 0x7f1338153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.769 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.751+0000 7f133d142700 1 -- 172.21.15.167:0/2565537355 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1334009500 con 0x7f1338159560 2022-11-27T16:14:47.769 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f133d142700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338159560 0x7f133815bfb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1328009fd0 tx=0x7f1328004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:47.769 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f13267fc700 1 -- 172.21.15.167:0/2565537355 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13280036a0 con 0x7f1338159560 2022-11-27T16:14:47.770 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f13267fc700 1 -- 172.21.15.167:0/2565537355 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132801f070 con 0x7f1338159560 2022-11-27T16:14:47.770 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f13267fc700 1 -- 172.21.15.167:0/2565537355 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1328003a60 con 0x7f1338159560 2022-11-27T16:14:47.770 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13280094a0 con 0x7f1338159560 2022-11-27T16:14:47.770 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13380c7db0 con 0x7f1338159560 2022-11-27T16:14:47.771 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f13267fc700 1 -- 172.21.15.167:0/2565537355 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f132800f050 con 0x7f1338159560 2022-11-27T16:14:47.772 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.752+0000 7f13267fc700 1 -- 172.21.15.167:0/2565537355 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f1328021070 con 0x7f1338159560 2022-11-27T16:14:47.772 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.753+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1338073080 con 0x7f1338159560 2022-11-27T16:14:47.774 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.756+0000 7f13267fc700 1 -- 172.21.15.167:0/2565537355 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f132802b020 con 0x7f1338159560 2022-11-27T16:14:47.842 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.839+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "94560d98-a6fa-494f-8407-d432c650a7f2", "id": 4} v 0) v1 -- 0x7f13380c7650 con 0x7f1338159560 2022-11-27T16:14:47.845 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.843+0000 7f13267fc700 1 -- 172.21.15.167:0/2565537355 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "94560d98-a6fa-494f-8407-d432c650a7f2", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f13380c7650 con 0x7f1338159560 2022-11-27T16:14:47.845 INFO:teuthology.orchestra.run.smithi167.stdout:4 2022-11-27T16:14:47.846 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338159560 msgr2=0x7f133815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:47.847 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338159560 0x7f133815bfb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1328009fd0 tx=0x7f1328004750 comp rx=0 tx=0).stop 2022-11-27T16:14:47.847 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 shutdown_connections 2022-11-27T16:14:47.847 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f133815dcf0 0x7f1338153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.847 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f13380cebe0 0x7f13380c64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.848 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 --2- 172.21.15.167:0/2565537355 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f1338159560 0x7f133815bfb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:47.848 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 >> 172.21.15.167:0/2565537355 conn(0x7f133806dee0 msgr2=0x7f1338158660 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:47.848 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 shutdown_connections 2022-11-27T16:14:47.848 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:47.844+0000 7f133fba7700 1 -- 172.21.15.167:0/2565537355 wait complete. 2022-11-27T16:14:47.858 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 815803f1-a95d-4921-a083-8185392357c6 5 2022-11-27T16:14:48.018 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.016+0000 7f0d63849700 1 Processor -- start 2022-11-27T16:14:48.018 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.016+0000 7f0d63849700 1 -- start start 2022-11-27T16:14:48.021 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d63849700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f0d5c159460 0x7f0d5c0693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.021 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d63849700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.022 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d63849700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d5c15dff0 con 0x7f0d5c0c9370 2022-11-27T16:14:48.022 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d63849700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d5c069910 con 0x7f0d5c159460 2022-11-27T16:14:48.022 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d63849700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c069b00 con 0x7f0d5c15db80 2022-11-27T16:14:48.023 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.023 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57912/0 (socket says 172.21.15.167:57912) 2022-11-27T16:14:48.023 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 -- 172.21.15.167:0/1316837325 learned_addr learned my addr 172.21.15.167:0/1316837325 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:48.024 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d615e5700 1 --2- 172.21.15.167:0/1316837325 >> v2:172.21.15.164:3300/0 conn(0x7f0d5c159460 0x7f0d5c0693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.024 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 -- 172.21.15.167:0/1316837325 >> v2:172.21.15.164:3300/0 conn(0x7f0d5c159460 msgr2=0x7f0d5c0693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:48.024 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 --2- 172.21.15.167:0/1316837325 >> v2:172.21.15.164:3300/0 conn(0x7f0d5c159460 0x7f0d5c0693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.024 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 -- 172.21.15.167:0/1316837325 >> v1:172.21.15.164:6789/0 conn(0x7f0d5c0c9370 legacy=0x7f0d5c0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:48.025 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 -- 172.21.15.167:0/1316837325 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c069de0 con 0x7f0d5c15db80 2022-11-27T16:14:48.025 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d60de4700 1 --2- 172.21.15.167:0/1316837325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cbf30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0098f0 tx=0x7f0d4c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8e52fbab9b04e7f server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.025 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.017+0000 7f0d4bfff700 1 -- 172.21.15.167:0/1316837325 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c004230 con 0x7f0d5c15db80 2022-11-27T16:14:48.025 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d4bfff700 1 -- 172.21.15.167:0/1316837325 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c004390 con 0x7f0d5c15db80 2022-11-27T16:14:48.025 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d4bfff700 1 -- 172.21.15.167:0/1316837325 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0036a0 con 0x7f0d5c15db80 2022-11-27T16:14:48.026 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 -- 172.21.15.167:0/1316837325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 msgr2=0x7f0d5c0cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.026 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 --2- 172.21.15.167:0/1316837325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cbf30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0098f0 tx=0x7f0d4c009c00 comp rx=0 tx=0).stop 2022-11-27T16:14:48.026 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 -- 172.21.15.167:0/1316837325 shutdown_connections 2022-11-27T16:14:48.026 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 --2- 172.21.15.167:0/1316837325 >> v2:172.21.15.164:3300/0 conn(0x7f0d5c159460 0x7f0d5c0693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.027 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 --2- 172.21.15.167:0/1316837325 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cbf30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.027 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 -- 172.21.15.167:0/1316837325 >> 172.21.15.167:0/1316837325 conn(0x7f0d5c06ce40 msgr2=0x7f0d5c06d240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.027 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 -- 172.21.15.167:0/1316837325 shutdown_connections 2022-11-27T16:14:48.027 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.018+0000 7f0d63849700 1 -- 172.21.15.167:0/1316837325 wait complete. 2022-11-27T16:14:48.030 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.020+0000 7f0d63849700 1 Processor -- start 2022-11-27T16:14:48.030 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.020+0000 7f0d63849700 1 -- start start 2022-11-27T16:14:48.031 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d63849700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c0c9370 0x7f0d5c1f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.031 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d63849700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c159460 0x7f0d5c0cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.031 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d63849700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.031 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d63849700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c153ad0 con 0x7f0d5c0c9370 2022-11-27T16:14:48.032 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d63849700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c0ce910 con 0x7f0d5c159460 2022-11-27T16:14:48.032 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d63849700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c0cd490 con 0x7f0d5c15db80 2022-11-27T16:14:48.032 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d61de6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.032 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d60de4700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c159460 0x7f0d5c0cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.033 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d61de6700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57920/0 (socket says 172.21.15.167:57920) 2022-11-27T16:14:48.033 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d615e5700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c0c9370 0x7f0d5c1f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.033 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d61de6700 1 -- 172.21.15.167:0/1554477602 learned_addr learned my addr 172.21.15.167:0/1554477602 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:48.034 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d60de4700 1 -- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 msgr2=0x7f0d5c0cce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.034 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d60de4700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.034 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d60de4700 1 -- 172.21.15.167:0/1554477602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c0c9370 msgr2=0x7f0d5c1f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.034 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d60de4700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c0c9370 0x7f0d5c1f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.034 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d60de4700 1 -- 172.21.15.167:0/1554477602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d4c009550 con 0x7f0d5c159460 2022-11-27T16:14:48.035 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d61de6700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:48.035 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.021+0000 7f0d60de4700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c159460 0x7f0d5c0cc8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c015000 tx=0x7f0d4c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.035 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.022+0000 7f0d49ffb700 1 -- 172.21.15.167:0/1554477602 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c0045d0 con 0x7f0d5c159460 2022-11-27T16:14:48.035 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.022+0000 7f0d49ffb700 1 -- 172.21.15.167:0/1554477602 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c00b660 con 0x7f0d5c159460 2022-11-27T16:14:48.036 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.022+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c0cd770 con 0x7f0d5c159460 2022-11-27T16:14:48.036 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.022+0000 7f0d49ffb700 1 -- 172.21.15.167:0/1554477602 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00ba10 con 0x7f0d5c159460 2022-11-27T16:14:48.036 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.022+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c0cdd30 con 0x7f0d5c159460 2022-11-27T16:14:48.036 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.023+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c073080 con 0x7f0d5c159460 2022-11-27T16:14:48.037 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.023+0000 7f0d49ffb700 1 -- 172.21.15.167:0/1554477602 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f0d4c017020 con 0x7f0d5c159460 2022-11-27T16:14:48.037 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.023+0000 7f0d49ffb700 1 -- 172.21.15.167:0/1554477602 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f0d4c011420 con 0x7f0d5c159460 2022-11-27T16:14:48.037 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.025+0000 7f0d49ffb700 1 -- 172.21.15.167:0/1554477602 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f0d4c00be60 con 0x7f0d5c159460 2022-11-27T16:14:48.142 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.140+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "815803f1-a95d-4921-a083-8185392357c6", "id": 5} v 0) v1 -- 0x7f0d5c073080 con 0x7f0d5c159460 2022-11-27T16:14:48.148 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.146+0000 7f0d49ffb700 1 -- 172.21.15.167:0/1554477602 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "815803f1-a95d-4921-a083-8185392357c6", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0d4c01b440 con 0x7f0d5c159460 2022-11-27T16:14:48.149 INFO:teuthology.orchestra.run.smithi167.stdout:5 2022-11-27T16:14:48.150 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c159460 msgr2=0x7f0d5c0cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.150 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c159460 0x7f0d5c0cc8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c015000 tx=0x7f0d4c004750 comp rx=0 tx=0).stop 2022-11-27T16:14:48.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 shutdown_connections 2022-11-27T16:14:48.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f0d5c15db80 0x7f0d5c0cce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f0d5c0c9370 0x7f0d5c1f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.151 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 --2- 172.21.15.167:0/1554477602 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f0d5c159460 0x7f0d5c0cc8f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.152 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 >> 172.21.15.167:0/1554477602 conn(0x7f0d5c06ce40 msgr2=0x7f0d5c0b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.152 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 shutdown_connections 2022-11-27T16:14:48.152 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.148+0000 7f0d63849700 1 -- 172.21.15.167:0/1554477602 wait complete. 2022-11-27T16:14:48.167 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 7e0d4655-210d-4601-8e1f-67a9fa7452ad 6 2022-11-27T16:14:48.198 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:48.195+0000 7ff5c49b9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:14:48.239 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:48.237+0000 7f30372cc200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:14:48.300 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.288+0000 7f3db549f700 1 Processor -- start 2022-11-27T16:14:48.301 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.288+0000 7f3db549f700 1 -- start start 2022-11-27T16:14:48.301 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.288+0000 7f3db549f700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3db00ae9a0 0x7f3db0069de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.288+0000 7f3db549f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00c9400 0x7f3db00c98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.288+0000 7f3db549f700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db00cdf30 con 0x7f3db00ae210 2022-11-27T16:14:48.302 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.288+0000 7f3db549f700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db006a320 con 0x7f3db00ae9a0 2022-11-27T16:14:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.288+0000 7f3db549f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db00aee10 con 0x7f3db00c9400 2022-11-27T16:14:48.303 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3daf7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00c9400 0x7f3db00c98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.304 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dae7fc700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f3db00ae9a0 0x7f3db0069de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.304 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3daf7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00c9400 0x7f3db00c98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57926/0 (socket says 172.21.15.167:57926) 2022-11-27T16:14:48.304 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3daf7fe700 1 -- 172.21.15.167:0/1188084364 learned_addr learned my addr 172.21.15.167:0/1188084364 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dae7fc700 1 -- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00c9400 msgr2=0x7f3db00c98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dae7fc700 1 --2- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00c9400 0x7f3db00c98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.305 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dae7fc700 1 -- 172.21.15.167:0/1188084364 >> v1:172.21.15.167:6789/0 conn(0x7f3db00ae210 legacy=0x7f3db00ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.306 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dae7fc700 1 -- 172.21.15.167:0/1188084364 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db00af0f0 con 0x7f3db00ae9a0 2022-11-27T16:14:48.306 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3daf7fe700 1 --2- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00c9400 0x7f3db00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:48.307 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dae7fc700 1 --2- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00ae9a0 0x7f3db0069de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009840 tx=0x7f3da4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7132a23ba1d637 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.307 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dadffb700 1 -- 172.21.15.167:0/1188084364 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4011830 con 0x7f3db00ae9a0 2022-11-27T16:14:48.307 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dadffb700 1 -- 172.21.15.167:0/1188084364 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da4011990 con 0x7f3db00ae9a0 2022-11-27T16:14:48.308 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.289+0000 7f3dadffb700 1 -- 172.21.15.167:0/1188084364 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da4011d20 con 0x7f3db00ae9a0 2022-11-27T16:14:48.308 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 -- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00ae9a0 msgr2=0x7f3db0069de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.309 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 --2- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00ae9a0 0x7f3db0069de0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009840 tx=0x7f3da4009b50 comp rx=0 tx=0).stop 2022-11-27T16:14:48.309 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 -- 172.21.15.167:0/1188084364 shutdown_connections 2022-11-27T16:14:48.309 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 --2- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00c9400 0x7f3db00c98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.310 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 --2- 172.21.15.167:0/1188084364 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00ae9a0 0x7f3db0069de0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.310 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 -- 172.21.15.167:0/1188084364 >> 172.21.15.167:0/1188084364 conn(0x7f3db006ae00 msgr2=0x7f3db00cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.311 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 -- 172.21.15.167:0/1188084364 shutdown_connections 2022-11-27T16:14:48.311 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.290+0000 7f3db549f700 1 -- 172.21.15.167:0/1188084364 wait complete. 2022-11-27T16:14:48.311 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.291+0000 7f3db549f700 1 Processor -- start 2022-11-27T16:14:48.312 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.291+0000 7f3db549f700 1 -- start start 2022-11-27T16:14:48.312 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3db549f700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00ae210 0x7f3db01f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.313 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3db549f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db00ae9a0 0x7f3db00c4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.313 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3db549f700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00c9400 0x7f3db00b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.314 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3db549f700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0148a40 con 0x7f3db00ae9a0 2022-11-27T16:14:48.314 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3db549f700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db01f1030 con 0x7f3db00c9400 2022-11-27T16:14:48.314 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3db549f700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db0147dc0 con 0x7f3db00ae210 2022-11-27T16:14:48.314 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3daf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00c9400 0x7f3db00b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.315 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3daeffd700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00ae210 0x7f3db01f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.315 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3daf7fe700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00c9400 0x7f3db00b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52368/0 (socket says 172.21.15.167:52368) 2022-11-27T16:14:48.315 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3daf7fe700 1 -- 172.21.15.167:0/4079136205 learned_addr learned my addr 172.21.15.167:0/4079136205 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:48.315 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.292+0000 7f3dae7fc700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db00ae9a0 0x7f3db00c4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.316 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3daf7fe700 1 -- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00ae210 msgr2=0x7f3db01f0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.316 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3daf7fe700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00ae210 0x7f3db01f0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.316 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3daf7fe700 1 -- 172.21.15.167:0/4079136205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db00ae9a0 msgr2=0x7f3db00c4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.317 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3daf7fe700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db00ae9a0 0x7f3db00c4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.317 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3daf7fe700 1 -- 172.21.15.167:0/4079136205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da8009b30 con 0x7f3db00c9400 2022-11-27T16:14:48.317 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3daeffd700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00ae210 0x7f3db01f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:48.317 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3daf7fe700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00c9400 0x7f3db00b0680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3da800efc0 tx=0x7f3da800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.318 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3d977fe700 1 -- 172.21.15.167:0/4079136205 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8018070 con 0x7f3db00c9400 2022-11-27T16:14:48.318 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3d977fe700 1 -- 172.21.15.167:0/4079136205 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da800f040 con 0x7f3db00c9400 2022-11-27T16:14:48.318 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da40094a0 con 0x7f3db00c9400 2022-11-27T16:14:48.318 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3d977fe700 1 -- 172.21.15.167:0/4079136205 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8003c40 con 0x7f3db00c9400 2022-11-27T16:14:48.319 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db0148380 con 0x7f3db00c9400 2022-11-27T16:14:48.319 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3d977fe700 1 -- 172.21.15.167:0/4079136205 <== mon.1 v2:172.21.15.167:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f3da800c880 con 0x7f3db00c9400 2022-11-27T16:14:48.319 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.293+0000 7f3d977fe700 1 -- 172.21.15.167:0/4079136205 <== mon.1 v2:172.21.15.167:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f3da801d410 con 0x7f3db00c9400 2022-11-27T16:14:48.319 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.294+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db0073080 con 0x7f3db00c9400 2022-11-27T16:14:48.319 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.296+0000 7f3d977fe700 1 -- 172.21.15.167:0/4079136205 <== mon.1 v2:172.21.15.167:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f3da80092e0 con 0x7f3db00c9400 2022-11-27T16:14:48.354 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:48.351+0000 7ff5c49b9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:14:48.381 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.379+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7e0d4655-210d-4601-8e1f-67a9fa7452ad", "id": 6} v 0) v1 -- 0x7f3db00cdbc0 con 0x7f3db00c9400 2022-11-27T16:14:48.387 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.385+0000 7f3d977fe700 1 -- 172.21.15.167:0/4079136205 <== mon.1 v2:172.21.15.167:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7e0d4655-210d-4601-8e1f-67a9fa7452ad", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3da802c0b0 con 0x7f3db00c9400 2022-11-27T16:14:48.388 INFO:teuthology.orchestra.run.smithi167.stdout:6 2022-11-27T16:14:48.389 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.386+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00c9400 msgr2=0x7f3db00b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.389 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.386+0000 7f3db549f700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00c9400 0x7f3db00b0680 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f3da800efc0 tx=0x7f3da800c3a0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.390 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.386+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 shutdown_connections 2022-11-27T16:14:48.390 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.386+0000 7f3db549f700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3db00c9400 0x7f3db00b0680 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.390 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.386+0000 7f3db549f700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f3db00ae210 0x7f3db01f0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.391 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.386+0000 7f3db549f700 1 --2- 172.21.15.167:0/4079136205 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3db00ae9a0 0x7f3db00c4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.391 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.386+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 >> 172.21.15.167:0/4079136205 conn(0x7f3db006ae00 msgr2=0x7f3db00ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.391 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.387+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 shutdown_connections 2022-11-27T16:14:48.392 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.387+0000 7f3db549f700 1 -- 172.21.15.167:0/4079136205 wait complete. 2022-11-27T16:14:48.400 DEBUG:teuthology.orchestra.run.smithi167:> sudo ceph --cluster ceph osd new 469ab427-7d6f-4f2a-9d32-275b76638d81 7 2022-11-27T16:14:48.401 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:48.399+0000 7f30372cc200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:14:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.520+0000 7f6e0e0ee700 1 Processor -- start 2022-11-27T16:14:48.523 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.521+0000 7f6e0e0ee700 1 -- start start 2022-11-27T16:14:48.535 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e0e0ee700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e08159560 0x7f6e080ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.535 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e0e0ee700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6e0815dcf0 0x7f6e080ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.536 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e0e0ee700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f6e080cebe0 0x7f6e080b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.536 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e0e0ee700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e080cade0 con 0x7f6e080cebe0 2022-11-27T16:14:48.537 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e0e0ee700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e080cd970 con 0x7f6e0815dcf0 2022-11-27T16:14:48.537 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e0e0ee700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e080cab00 con 0x7f6e08159560 2022-11-27T16:14:48.538 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6e0815dcf0 0x7f6e080ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.538 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e077fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e08159560 0x7f6e080ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.538 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f6e0815dcf0 0x7f6e080ce6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.167:52372/0 (socket says 172.21.15.167:52372) 2022-11-27T16:14:48.539 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e077fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e08159560 0x7f6e080ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57934/0 (socket says 172.21.15.167:57934) 2022-11-27T16:14:48.539 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 -- 172.21.15.167:0/111040553 learned_addr learned my addr 172.21.15.167:0/111040553 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:48.540 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e07fff700 1 --2- 172.21.15.167:0/111040553 >> v2:172.21.15.164:3300/0 conn(0x7f6e080cebe0 0x7f6e080b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.540 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 -- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e08159560 msgr2=0x7f6e080ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:48.541 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 --2- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e08159560 0x7f6e080ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.541 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 -- 172.21.15.167:0/111040553 >> v2:172.21.15.164:3300/0 conn(0x7f6e080cebe0 msgr2=0x7f6e080b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:48.542 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 --2- 172.21.15.167:0/111040553 >> v2:172.21.15.164:3300/0 conn(0x7f6e080cebe0 0x7f6e080b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.542 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 -- 172.21.15.167:0/111040553 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e080cdbc0 con 0x7f6e0815dcf0 2022-11-27T16:14:48.542 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e077fe700 1 --2- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e08159560 0x7f6e080ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:48.543 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.522+0000 7f6e06ffd700 1 --2- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e0815dcf0 0x7f6e080ce6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e080cd3c0 tx=0x7f6df0009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38842cc1f91cf359 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.543 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e067fc700 1 -- 172.21.15.167:0/111040553 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0011830 con 0x7f6e0815dcf0 2022-11-27T16:14:48.544 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e067fc700 1 -- 172.21.15.167:0/111040553 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df0011990 con 0x7f6e0815dcf0 2022-11-27T16:14:48.544 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e067fc700 1 -- 172.21.15.167:0/111040553 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df0011d20 con 0x7f6e0815dcf0 2022-11-27T16:14:48.545 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e0815dcf0 msgr2=0x7f6e080ce6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.545 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e0815dcf0 0x7f6e080ce6a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6e080cd3c0 tx=0x7f6df0009bd0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.546 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/111040553 shutdown_connections 2022-11-27T16:14:48.546 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/111040553 >> v2:172.21.15.164:3300/0 conn(0x7f6e080cebe0 0x7f6e080b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.547 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e08159560 0x7f6e080ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.547 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/111040553 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e0815dcf0 0x7f6e080ce6a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.548 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/111040553 >> 172.21.15.167:0/111040553 conn(0x7f6e0806dee0 msgr2=0x7f6e0806e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.548 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/111040553 shutdown_connections 2022-11-27T16:14:48.548 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.523+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/111040553 wait complete. 2022-11-27T16:14:48.549 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.524+0000 7f6e0e0ee700 1 Processor -- start 2022-11-27T16:14:48.549 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e0e0ee700 1 -- start start 2022-11-27T16:14:48.550 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e0e0ee700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e080cebe0 0x7f6e081514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.550 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e0e0ee700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08159560 0x7f6e0806be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.551 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e0e0ee700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e0815dcf0 0x7f6e08153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.551 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e0e0ee700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0815ab20 con 0x7f6e080cebe0 2022-11-27T16:14:48.552 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e0e0ee700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0815ae00 con 0x7f6e08159560 2022-11-27T16:14:48.552 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e0e0ee700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0815b0e0 con 0x7f6e0815dcf0 2022-11-27T16:14:48.553 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e07fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e0815dcf0 0x7f6e08153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.553 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e07fff700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e0815dcf0 0x7f6e08153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.167:57944/0 (socket says 172.21.15.167:57944) 2022-11-27T16:14:48.554 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e06ffd700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08159560 0x7f6e0806be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.554 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.525+0000 7f6e07fff700 1 -- 172.21.15.167:0/532157519 learned_addr learned my addr 172.21.15.167:0/532157519 (peer_addr_for_me v2:172.21.15.167:0/0) 2022-11-27T16:14:48.555 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e077fe700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e080cebe0 0x7f6e081514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.555 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e07fff700 1 -- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08159560 msgr2=0x7f6e0806be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.555 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e07fff700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08159560 0x7f6e0806be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.556 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e07fff700 1 -- 172.21.15.167:0/532157519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e080cebe0 msgr2=0x7f6e081514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.556 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e07fff700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e080cebe0 0x7f6e081514e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.557 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e07fff700 1 -- 172.21.15.167:0/532157519 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8009430 con 0x7f6e0815dcf0 2022-11-27T16:14:48.557 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e06ffd700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08159560 0x7f6e0806be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:48.558 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e077fe700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e080cebe0 0x7f6e081514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:14:48.558 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e07fff700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e0815dcf0 0x7f6e08153590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000efc0 tx=0x7f6e0000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.559 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6dfffff700 1 -- 172.21.15.167:0/532157519 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e0000cc90 con 0x7f6e0815dcf0 2022-11-27T16:14:48.559 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6dfffff700 1 -- 172.21.15.167:0/532157519 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e0000f040 con 0x7f6e0815dcf0 2022-11-27T16:14:48.560 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6df0009550 con 0x7f6e0815dcf0 2022-11-27T16:14:48.560 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6dfffff700 1 -- 172.21.15.167:0/532157519 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e000146c0 con 0x7f6e0815dcf0 2022-11-27T16:14:48.560 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0815b6a0 con 0x7f6e0815dcf0 2022-11-27T16:14:48.561 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6dfffff700 1 -- 172.21.15.167:0/532157519 <== mon.2 v2:172.21.15.167:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f6e00014820 con 0x7f6e0815dcf0 2022-11-27T16:14:48.561 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.526+0000 7f6dfffff700 1 -- 172.21.15.167:0/532157519 <== mon.2 v2:172.21.15.167:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f6e0001dce0 con 0x7f6e0815dcf0 2022-11-27T16:14:48.562 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.527+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0815b8c0 con 0x7f6e0815dcf0 2022-11-27T16:14:48.562 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.531+0000 7f6dfffff700 1 -- 172.21.15.167:0/532157519 <== mon.2 v2:172.21.15.167:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f6e0003b6d0 con 0x7f6e0815dcf0 2022-11-27T16:14:48.616 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.614+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_command({"prefix": "osd new", "uuid": "469ab427-7d6f-4f2a-9d32-275b76638d81", "id": 7} v 0) v1 -- 0x7f6e080cb590 con 0x7f6e0815dcf0 2022-11-27T16:14:48.627 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.625+0000 7f6dfffff700 1 -- 172.21.15.167:0/532157519 <== mon.2 v2:172.21.15.167:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "469ab427-7d6f-4f2a-9d32-275b76638d81", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6e0002a070 con 0x7f6e0815dcf0 2022-11-27T16:14:48.627 INFO:teuthology.orchestra.run.smithi167.stdout:7 2022-11-27T16:14:48.629 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e0815dcf0 msgr2=0x7f6e08153590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.629 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e0815dcf0 0x7f6e08153590 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e0000efc0 tx=0x7f6e0000c3a0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.629 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 shutdown_connections 2022-11-27T16:14:48.630 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f6e0815dcf0 0x7f6e08153590 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.630 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f6e080cebe0 0x7f6e081514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.630 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 --2- 172.21.15.167:0/532157519 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f6e08159560 0x7f6e0806be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.630 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 >> 172.21.15.167:0/532157519 conn(0x7f6e0806dee0 msgr2=0x7f6e080cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.631 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 shutdown_connections 2022-11-27T16:14:48.631 INFO:teuthology.orchestra.run.smithi167.stderr:2022-11-27T16:14:48.626+0000 7f6e0e0ee700 1 -- 172.21.15.167:0/532157519 wait complete. 2022-11-27T16:14:48.640 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T16:14:48.640 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T16:14:48.643 INFO:tasks.ceph.osd.0:Started 2022-11-27T16:14:48.643 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T16:14:48.643 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T16:14:48.646 INFO:tasks.ceph.osd.1:Started 2022-11-27T16:14:48.646 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T16:14:48.646 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T16:14:48.649 INFO:tasks.ceph.osd.2:Started 2022-11-27T16:14:48.649 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T16:14:48.650 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T16:14:48.652 INFO:tasks.ceph.osd.3:Started 2022-11-27T16:14:48.652 INFO:tasks.ceph.osd.4:Restarting daemon 2022-11-27T16:14:48.653 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-11-27T16:14:48.655 INFO:tasks.ceph.osd.4:Started 2022-11-27T16:14:48.656 INFO:tasks.ceph.osd.5:Restarting daemon 2022-11-27T16:14:48.656 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-11-27T16:14:48.658 INFO:tasks.ceph.osd.5:Started 2022-11-27T16:14:48.658 INFO:tasks.ceph.osd.6:Restarting daemon 2022-11-27T16:14:48.659 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-11-27T16:14:48.661 INFO:tasks.ceph.osd.6:Started 2022-11-27T16:14:48.661 INFO:tasks.ceph.osd.7:Restarting daemon 2022-11-27T16:14:48.662 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-11-27T16:14:48.664 INFO:tasks.ceph.osd.7:Started 2022-11-27T16:14:48.665 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:14:48.835 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.833+0000 7f39082ad700 1 Processor -- start 2022-11-27T16:14:48.836 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.833+0000 7f39082ad700 1 -- start start 2022-11-27T16:14:48.842 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f39082ad700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3900157cd0 0x7f3900071160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.842 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f39082ad700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f39000c9020 0x7f390006c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.842 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f39082ad700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39001580a0 con 0x7f390006ca00 2022-11-27T16:14:48.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f39082ad700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000ca4b0 con 0x7f3900157cd0 2022-11-27T16:14:48.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f39082ad700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3900069790 con 0x7f39000c9020 2022-11-27T16:14:48.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3906049700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f3900157cd0 0x7f3900071160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f390684a700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f390006ca00 0x7f390015b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59220/0 (socket says 172.21.15.164:59220) 2022-11-27T16:14:48.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f390684a700 1 -- 172.21.15.164:0/4206922619 learned_addr learned my addr 172.21.15.164:0/4206922619 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:14:48.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3905848700 1 --2- 172.21.15.164:0/4206922619 >> v2:172.21.15.167:3300/0 conn(0x7f39000c9020 0x7f390006c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3906049700 1 -- 172.21.15.164:0/4206922619 >> v2:172.21.15.167:3300/0 conn(0x7f39000c9020 msgr2=0x7f390006c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:48.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3906049700 1 --2- 172.21.15.164:0/4206922619 >> v2:172.21.15.167:3300/0 conn(0x7f39000c9020 0x7f390006c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.845 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3906049700 1 -- 172.21.15.164:0/4206922619 >> v1:172.21.15.164:6789/0 conn(0x7f390006ca00 legacy=0x7f390015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.845 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3906049700 1 -- 172.21.15.164:0/4206922619 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39000c98a0 con 0x7f3900157cd0 2022-11-27T16:14:48.845 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3905047700 1 -- 172.21.15.164:0/4206922619 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332753288 0 0) 0x7f39001580a0 con 0x7f390006ca00 2022-11-27T16:14:48.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.834+0000 7f3906049700 1 --2- 172.21.15.164:0/4206922619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3900157cd0 0x7f3900071160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f39000cf510 tx=0x7f38fc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4fa8d7b07079ee server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f3905047700 1 -- 172.21.15.164:0/4206922619 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc011830 con 0x7f3900157cd0 2022-11-27T16:14:48.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f3905047700 1 -- 172.21.15.164:0/4206922619 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc011990 con 0x7f3900157cd0 2022-11-27T16:14:48.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f3905047700 1 -- 172.21.15.164:0/4206922619 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc011d20 con 0x7f3900157cd0 2022-11-27T16:14:48.847 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 -- 172.21.15.164:0/4206922619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3900157cd0 msgr2=0x7f3900071160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.847 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 --2- 172.21.15.164:0/4206922619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3900157cd0 0x7f3900071160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f39000cf510 tx=0x7f38fc009b20 comp rx=0 tx=0).stop 2022-11-27T16:14:48.847 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 -- 172.21.15.164:0/4206922619 shutdown_connections 2022-11-27T16:14:48.847 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 --2- 172.21.15.164:0/4206922619 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f3900157cd0 0x7f3900071160 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.848 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 --2- 172.21.15.164:0/4206922619 >> v2:172.21.15.167:3300/0 conn(0x7f39000c9020 0x7f390006c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.848 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 -- 172.21.15.164:0/4206922619 >> 172.21.15.164:0/4206922619 conn(0x7f39000ce410 msgr2=0x7f39000ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.848 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 -- 172.21.15.164:0/4206922619 shutdown_connections 2022-11-27T16:14:48.849 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.835+0000 7f39082ad700 1 -- 172.21.15.164:0/4206922619 wait complete. 2022-11-27T16:14:48.849 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.836+0000 7f39082ad700 1 Processor -- start 2022-11-27T16:14:48.849 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f39082ad700 1 -- start start 2022-11-27T16:14:48.850 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f39082ad700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f390006ca00 0x7f39000c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.850 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f39082ad700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39000c9020 0x7f39001421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.850 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f39082ad700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900157cd0 0x7f39000b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:48.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f39082ad700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39000c5a20 con 0x7f390006ca00 2022-11-27T16:14:48.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f39082ad700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900141a30 con 0x7f3900157cd0 2022-11-27T16:14:48.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f39082ad700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39000c5c50 con 0x7f39000c9020 2022-11-27T16:14:48.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f3906049700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f390006ca00 0x7f39000c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f3906049700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f390006ca00 0x7f39000c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36422/0 (socket says 172.21.15.164:36422) 2022-11-27T16:14:48.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f3906049700 1 -- 172.21.15.164:0/1481400278 learned_addr learned my addr 172.21.15.164:0/1481400278 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:14:48.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f390684a700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900157cd0 0x7f39000b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.837+0000 7f3905848700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39000c9020 0x7f39001421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:48.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f3906049700 1 -- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39000c9020 msgr2=0x7f39001421d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f3906049700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39000c9020 0x7f39001421d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f3906049700 1 -- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900157cd0 msgr2=0x7f39000b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f3906049700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900157cd0 0x7f39000b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f3906049700 1 -- 172.21.15.164:0/1481400278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc0094a0 con 0x7f390006ca00 2022-11-27T16:14:48.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f3906049700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f390006ca00 0x7f39000c54e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0090b0 tx=0x7f38fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:48.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f38eeffd700 1 -- 172.21.15.164:0/1481400278 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0036a0 con 0x7f390006ca00 2022-11-27T16:14:48.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f38eeffd700 1 -- 172.21.15.164:0/1481400278 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc01f070 con 0x7f390006ca00 2022-11-27T16:14:48.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f38eeffd700 1 -- 172.21.15.164:0/1481400278 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc003a60 con 0x7f390006ca00 2022-11-27T16:14:48.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39000c5f30 con 0x7f390006ca00 2022-11-27T16:14:48.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3900141d80 con 0x7f390006ca00 2022-11-27T16:14:48.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f38eeffd700 1 -- 172.21.15.164:0/1481400278 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f38fc00f050 con 0x7f390006ca00 2022-11-27T16:14:48.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.838+0000 7f38eeffd700 1 -- 172.21.15.164:0/1481400278 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f38fc021070 con 0x7f390006ca00 2022-11-27T16:14:48.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.839+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900142040 con 0x7f390006ca00 2022-11-27T16:14:48.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.843+0000 7f38eeffd700 1 -- 172.21.15.164:0/1481400278 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f38fc0466d0 con 0x7f390006ca00 2022-11-27T16:14:48.915 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.912+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3900141ba0 con 0x7f390006ca00 2022-11-27T16:14:48.916 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.913+0000 7f38eeffd700 1 -- 172.21.15.164:0/1481400278 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f38fc035070 con 0x7f390006ca00 2022-11-27T16:14:48.916 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:14:48.917 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":10,"fsid":"94c9dd9b-1805-4f9e-833a-3b29514744e3","created":"2022-11-27T16:14:46.055229+0000","modified":"2022-11-27T16:14:48.616289+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:14:48.616289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"fc87cc2c-4aba-41f8-9b94-8843abd34341","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2288745b-11fa-433b-bf88-dbfd0693401d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"82ed3e90-5d84-4fc3-ac13-a6a146bc60f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"864ca05d-4b9e-4111-8130-dd230c4a663c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"94560d98-a6fa-494f-8407-d432c650a7f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"815803f1-a95d-4921-a083-8185392357c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7e0d4655-210d-4601-8e1f-67a9fa7452ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"469ab427-7d6f-4f2a-9d32-275b76638d81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:14:48.918 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f390006ca00 msgr2=0x7f39000c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:48.918 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f390006ca00 0x7f39000c54e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f38fc0090b0 tx=0x7f38fc004750 comp rx=0 tx=0).stop 2022-11-27T16:14:48.919 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 shutdown_connections 2022-11-27T16:14:48.919 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f3900157cd0 0x7f39000b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.919 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f390006ca00 0x7f39000c54e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.919 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 --2- 172.21.15.164:0/1481400278 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f39000c9020 0x7f39001421d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:48.920 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 >> 172.21.15.164:0/1481400278 conn(0x7f39000ce410 msgr2=0x7f390006a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:48.920 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 shutdown_connections 2022-11-27T16:14:48.920 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:14:48.914+0000 7f39082ad700 1 -- 172.21.15.164:0/1481400278 wait complete. 2022-11-27T16:14:48.929 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T16:14:48.930 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T16:14:49.005 INFO:tasks.ceph.osd.0.smithi164.stderr:2022-11-27T16:14:49.002+0000 7f9258aef5c0 -1 Falling back to public interface 2022-11-27T16:14:49.102 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:49.099+0000 7ff5c49b9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:14:49.117 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:49.115+0000 7f30372cc200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:14:49.177 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:49.175+0000 7ff5c49b9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:14:49.193 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:49.191+0000 7f30372cc200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:14:49.263 INFO:tasks.ceph.osd.7.smithi167.stderr:2022-11-27T16:14:49.261+0000 7ff141cae5c0 -1 Falling back to public interface 2022-11-27T16:14:49.264 INFO:tasks.ceph.osd.4.smithi167.stderr:2022-11-27T16:14:49.262+0000 7f507c0af5c0 -1 Falling back to public interface 2022-11-27T16:14:49.273 INFO:tasks.ceph.osd.1.smithi164.stderr:2022-11-27T16:14:49.271+0000 7fbd444825c0 -1 Falling back to public interface 2022-11-27T16:14:49.274 INFO:tasks.ceph.osd.3.smithi164.stderr:2022-11-27T16:14:49.271+0000 7f849b8c45c0 -1 Falling back to public interface 2022-11-27T16:14:49.274 INFO:tasks.ceph.osd.3.smithi164.stderr:2022-11-27T16:14:49.271+0000 7f8492c82700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2022-11-27T16:14:49.277 INFO:tasks.ceph.osd.2.smithi164.stderr:2022-11-27T16:14:49.275+0000 7f57eb7ce5c0 -1 Falling back to public interface 2022-11-27T16:14:49.294 INFO:tasks.ceph.osd.6.smithi167.stderr:2022-11-27T16:14:49.292+0000 7f47ec2635c0 -1 Falling back to public interface 2022-11-27T16:14:49.295 INFO:tasks.ceph.osd.5.smithi167.stderr:2022-11-27T16:14:49.292+0000 7f224a7495c0 -1 Falling back to public interface 2022-11-27T16:14:49.295 INFO:tasks.ceph.osd.5.smithi167.stderr:2022-11-27T16:14:49.293+0000 7f2241b07700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2022-11-27T16:14:49.296 INFO:tasks.ceph.osd.6.smithi167.stderr:2022-11-27T16:14:49.294+0000 7f47e3621700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6825/0: (98) Address already in use 2022-11-27T16:14:49.319 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:49.316+0000 7ff5c49b9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:14:49.332 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:49.331+0000 7f30372cc200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:14:49.476 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:49.473+0000 7ff5c49b9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:14:49.489 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:49.488+0000 7f30372cc200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:14:49.608 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:49.606+0000 7ff5c49b9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:14:49.623 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:49.621+0000 7f30372cc200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:14:49.736 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:14:49.849 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.847+0000 7fd131ae7700 1 Processor -- start 2022-11-27T16:14:49.850 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.847+0000 7fd131ae7700 1 -- start start 2022-11-27T16:14:49.853 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd131ae7700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd12c157b70 0x7fd12c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:49.853 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd131ae7700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fd12c06db20 0x7fd12c06df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:49.853 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd131ae7700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12c06e480 con 0x7fd12c15c1b0 2022-11-27T16:14:49.854 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd131ae7700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c0c94f0 con 0x7fd12c157b70 2022-11-27T16:14:49.854 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd131ae7700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd12c069570 con 0x7fd12c06db20 2022-11-27T16:14:49.854 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12b7fe700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fd12c157b70 0x7fd12c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:49.854 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12affd700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fd12c15c1b0 0x7fd12c0c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59230/0 (socket says 172.21.15.164:59230) 2022-11-27T16:14:49.855 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12affd700 1 -- 172.21.15.164:0/1719099768 learned_addr learned my addr 172.21.15.164:0/1719099768 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:14:49.855 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12bfff700 1 --2- 172.21.15.164:0/1719099768 >> v2:172.21.15.167:3300/0 conn(0x7fd12c06db20 0x7fd12c06df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:49.855 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12b7fe700 1 -- 172.21.15.164:0/1719099768 >> v2:172.21.15.167:3300/0 conn(0x7fd12c06db20 msgr2=0x7fd12c06df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:14:49.855 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12b7fe700 1 --2- 172.21.15.164:0/1719099768 >> v2:172.21.15.167:3300/0 conn(0x7fd12c06db20 0x7fd12c06df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.856 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12b7fe700 1 -- 172.21.15.164:0/1719099768 >> v1:172.21.15.164:6789/0 conn(0x7fd12c15c1b0 legacy=0x7fd12c0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:49.856 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12a7fc700 1 -- 172.21.15.164:0/1719099768 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758995262 0 0) 0x7fd12c06e480 con 0x7fd12c15c1b0 2022-11-27T16:14:49.856 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12b7fe700 1 -- 172.21.15.164:0/1719099768 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c069850 con 0x7fd12c157b70 2022-11-27T16:14:49.856 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12bfff700 1 --2- 172.21.15.164:0/1719099768 >> v2:172.21.15.167:3300/0 conn(0x7fd12c06db20 0x7fd12c06df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:14:49.856 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12b7fe700 1 --2- 172.21.15.164:0/1719099768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c157b70 0x7fd12c0cc970 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06fe00 tx=0x7fd11c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7adf66d3d634c09 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:49.857 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12a7fc700 1 -- 172.21.15.164:0/1719099768 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c011830 con 0x7fd12c157b70 2022-11-27T16:14:49.857 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12a7fc700 1 -- 172.21.15.164:0/1719099768 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11c011990 con 0x7fd12c157b70 2022-11-27T16:14:49.857 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.848+0000 7fd12a7fc700 1 -- 172.21.15.164:0/1719099768 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11c011d20 con 0x7fd12c157b70 2022-11-27T16:14:49.857 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 -- 172.21.15.164:0/1719099768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c157b70 msgr2=0x7fd12c0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:49.858 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 --2- 172.21.15.164:0/1719099768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c157b70 0x7fd12c0cc970 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd12c06fe00 tx=0x7fd11c009b20 comp rx=0 tx=0).stop 2022-11-27T16:14:49.858 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 -- 172.21.15.164:0/1719099768 shutdown_connections 2022-11-27T16:14:49.858 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 --2- 172.21.15.164:0/1719099768 >> v2:172.21.15.167:3300/0 conn(0x7fd12c06db20 0x7fd12c06df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.858 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 --2- 172.21.15.164:0/1719099768 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c157b70 0x7fd12c0cc970 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.858 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 -- 172.21.15.164:0/1719099768 >> 172.21.15.164:0/1719099768 conn(0x7fd12c0ce390 msgr2=0x7fd12c0ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:49.859 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 -- 172.21.15.164:0/1719099768 shutdown_connections 2022-11-27T16:14:49.859 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.849+0000 7fd131ae7700 1 -- 172.21.15.164:0/1719099768 wait complete. 2022-11-27T16:14:49.859 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.850+0000 7fd131ae7700 1 Processor -- start 2022-11-27T16:14:49.859 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.850+0000 7fd131ae7700 1 -- start start 2022-11-27T16:14:49.865 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd131ae7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd12c06db20 0x7fd12c0c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:49.865 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd131ae7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd12c157b70 0x7fd12c0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:49.866 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd131ae7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c15c1b0 0x7fd12c1f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:49.866 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd131ae7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0aefa0 con 0x7fd12c15c1b0 2022-11-27T16:14:49.866 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd131ae7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0aed20 con 0x7fd12c06db20 2022-11-27T16:14:49.867 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd131ae7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c0c2060 con 0x7fd12c157b70 2022-11-27T16:14:49.867 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd12bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c15c1b0 0x7fd12c1f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:49.868 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd12bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c15c1b0 0x7fd12c1f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36438/0 (socket says 172.21.15.164:36438) 2022-11-27T16:14:49.868 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd12bfff700 1 -- 172.21.15.164:0/1612689437 learned_addr learned my addr 172.21.15.164:0/1612689437 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:14:49.869 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd12b7fe700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd12c06db20 0x7fd12c0c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:49.869 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.851+0000 7fd12affd700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd12c157b70 0x7fd12c0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:49.869 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd12bfff700 1 -- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd12c157b70 msgr2=0x7fd12c0ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:49.870 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd12bfff700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd12c157b70 0x7fd12c0ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.870 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd12bfff700 1 -- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd12c06db20 msgr2=0x7fd12c0c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:49.871 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd12bfff700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd12c06db20 0x7fd12c0c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.871 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd12bfff700 1 -- 172.21.15.164:0/1612689437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c0094a0 con 0x7fd12c15c1b0 2022-11-27T16:14:49.871 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd12bfff700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c15c1b0 0x7fd12c1f2360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd124007db0 tx=0x7fd12400d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:49.872 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd113fff700 1 -- 172.21.15.164:0/1612689437 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400f040 con 0x7fd12c15c1b0 2022-11-27T16:14:49.872 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd113fff700 1 -- 172.21.15.164:0/1612689437 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124003c20 con 0x7fd12c15c1b0 2022-11-27T16:14:49.873 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12c0c2340 con 0x7fd12c15c1b0 2022-11-27T16:14:49.873 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd113fff700 1 -- 172.21.15.164:0/1612689437 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400a2e0 con 0x7fd12c15c1b0 2022-11-27T16:14:49.873 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12c0c2900 con 0x7fd12c15c1b0 2022-11-27T16:14:49.874 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd113fff700 1 -- 172.21.15.164:0/1612689437 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fd12400d900 con 0x7fd12c15c1b0 2022-11-27T16:14:49.874 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.852+0000 7fd113fff700 1 -- 172.21.15.164:0/1612689437 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fd124008b10 con 0x7fd12c15c1b0 2022-11-27T16:14:49.874 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.853+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd12c0c2ac0 con 0x7fd12c15c1b0 2022-11-27T16:14:49.874 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.855+0000 7fd113fff700 1 -- 172.21.15.164:0/1612689437 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fd12400dd70 con 0x7fd12c15c1b0 2022-11-27T16:14:49.927 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.925+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd12c0c2c50 con 0x7fd12c15c1b0 2022-11-27T16:14:49.928 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.925+0000 7fd113fff700 1 -- 172.21.15.164:0/1612689437 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7fd124008d70 con 0x7fd12c15c1b0 2022-11-27T16:14:49.928 INFO:teuthology.misc.health.smithi164.stdout: 2022-11-27T16:14:49.928 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":10,"fsid":"94c9dd9b-1805-4f9e-833a-3b29514744e3","created":"2022-11-27T16:14:46.055229+0000","modified":"2022-11-27T16:14:48.616289+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:14:48.616289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"fc87cc2c-4aba-41f8-9b94-8843abd34341","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2288745b-11fa-433b-bf88-dbfd0693401d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"82ed3e90-5d84-4fc3-ac13-a6a146bc60f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"864ca05d-4b9e-4111-8130-dd230c4a663c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"94560d98-a6fa-494f-8407-d432c650a7f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"815803f1-a95d-4921-a083-8185392357c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7e0d4655-210d-4601-8e1f-67a9fa7452ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"469ab427-7d6f-4f2a-9d32-275b76638d81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:14:49.930 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c15c1b0 msgr2=0x7fd12c1f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:49.930 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c15c1b0 0x7fd12c1f2360 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd124007db0 tx=0x7fd12400d3a0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.930 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 shutdown_connections 2022-11-27T16:14:49.930 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fd12c15c1b0 0x7fd12c1f2360 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.930 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fd12c06db20 0x7fd12c0c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.931 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 --2- 172.21.15.164:0/1612689437 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fd12c157b70 0x7fd12c0ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:49.931 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 >> 172.21.15.164:0/1612689437 conn(0x7fd12c0ce390 msgr2=0x7fd12c156d80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:49.931 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 shutdown_connections 2022-11-27T16:14:49.932 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:49.927+0000 7fd131ae7700 1 -- 172.21.15.164:0/1612689437 wait complete. 2022-11-27T16:14:49.944 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T16:14:49.973 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:49.970+0000 7ff5c49b9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:14:49.974 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:49.972+0000 7f30372cc200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:14:50.126 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:50.123+0000 7ff5c49b9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:14:50.127 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:50.125+0000 7f30372cc200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:14:50.240 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:50.238+0000 7ff5c49b9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:14:50.242 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:50.240+0000 7f30372cc200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:14:50.386 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:50.384+0000 7ff5c49b9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:14:50.387 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:50.385+0000 7f30372cc200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:14:50.484 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:50.482+0000 7ff5c49b9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:14:50.486 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:50.485+0000 7f30372cc200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:14:50.842 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:50.838+0000 7ff5c49b9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:14:50.843 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:50.842+0000 7f30372cc200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:14:51.418 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:51.416+0000 7ff5c49b9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:14:51.421 INFO:tasks.ceph.mgr.x.smithi167.stderr:2022-11-27T16:14:51.420+0000 7f30372cc200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:14:52.444 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:52.439+0000 7ff57cd83700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-11-27T16:14:52.444 INFO:tasks.ceph.mgr.y.smithi164.stderr:2022-11-27T16:14:52.439+0000 7ff57cd83700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-11-27T16:14:52.543 INFO:tasks.ceph.osd.0.smithi164.stderr:2022-11-27T16:14:52.541+0000 7f9258aef5c0 -1 osd.0 0 log_to_monitors true 2022-11-27T16:14:52.778 INFO:tasks.ceph.osd.1.smithi164.stderr:2022-11-27T16:14:52.775+0000 7fbd444825c0 -1 osd.1 0 log_to_monitors true 2022-11-27T16:14:52.801 INFO:tasks.ceph.osd.2.smithi164.stderr:2022-11-27T16:14:52.799+0000 7f57eb7ce5c0 -1 osd.2 0 log_to_monitors true 2022-11-27T16:14:52.811 INFO:tasks.ceph.osd.3.smithi164.stderr:2022-11-27T16:14:52.809+0000 7f849b8c45c0 -1 osd.3 0 log_to_monitors true 2022-11-27T16:14:52.821 INFO:tasks.ceph.osd.7.smithi167.stderr:2022-11-27T16:14:52.819+0000 7ff141cae5c0 -1 osd.7 0 log_to_monitors true 2022-11-27T16:14:52.826 INFO:tasks.ceph.osd.5.smithi167.stderr:2022-11-27T16:14:52.823+0000 7f224a7495c0 -1 osd.5 0 log_to_monitors true 2022-11-27T16:14:52.827 INFO:tasks.ceph.osd.6.smithi167.stderr:2022-11-27T16:14:52.823+0000 7f47ec2635c0 -1 osd.6 0 log_to_monitors true 2022-11-27T16:14:52.830 INFO:tasks.ceph.osd.4.smithi167.stderr:2022-11-27T16:14:52.829+0000 7f507c0af5c0 -1 osd.4 0 log_to_monitors true 2022-11-27T16:14:56.275 INFO:tasks.ceph.osd.4.smithi167.stderr:2022-11-27T16:14:56.273+0000 7f506fc66700 -1 osd.4 0 waiting for initial osdmap 2022-11-27T16:14:56.291 INFO:tasks.ceph.osd.4.smithi167.stderr:2022-11-27T16:14:56.288+0000 7f5068041700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.339 INFO:tasks.ceph.osd.1.smithi164.stderr:2022-11-27T16:14:56.336+0000 7fbd38039700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T16:14:56.353 INFO:tasks.ceph.osd.1.smithi164.stderr:2022-11-27T16:14:56.350+0000 7fbd30414700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.421 INFO:tasks.ceph.osd.5.smithi167.stderr:2022-11-27T16:14:56.419+0000 7f223e300700 -1 osd.5 0 waiting for initial osdmap 2022-11-27T16:14:56.434 INFO:tasks.ceph.osd.5.smithi167.stderr:2022-11-27T16:14:56.432+0000 7f22366db700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.447 INFO:tasks.ceph.osd.7.smithi167.stderr:2022-11-27T16:14:56.446+0000 7ff135865700 -1 osd.7 0 waiting for initial osdmap 2022-11-27T16:14:56.459 INFO:tasks.ceph.osd.2.smithi164.stderr:2022-11-27T16:14:56.455+0000 7f57df385700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T16:14:56.459 INFO:tasks.ceph.osd.7.smithi167.stderr:2022-11-27T16:14:56.456+0000 7ff12dc40700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.470 INFO:tasks.ceph.osd.6.smithi167.stderr:2022-11-27T16:14:56.468+0000 7f47dfe1a700 -1 osd.6 0 waiting for initial osdmap 2022-11-27T16:14:56.471 INFO:tasks.ceph.osd.2.smithi164.stderr:2022-11-27T16:14:56.468+0000 7f57d7760700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.478 INFO:tasks.ceph.osd.6.smithi167.stderr:2022-11-27T16:14:56.477+0000 7f47d81f5700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.500 INFO:tasks.ceph.osd.3.smithi164.stderr:2022-11-27T16:14:56.497+0000 7f848f47b700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T16:14:56.507 INFO:tasks.ceph.osd.0.smithi164.stderr:2022-11-27T16:14:56.505+0000 7f924c6a6700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T16:14:56.514 INFO:tasks.ceph.osd.3.smithi164.stderr:2022-11-27T16:14:56.512+0000 7f8487856700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.523 INFO:tasks.ceph.osd.0.smithi164.stderr:2022-11-27T16:14:56.521+0000 7f9244a81700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:14:56.754 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:14:56.865 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.863+0000 7ff484499700 1 Processor -- start 2022-11-27T16:14:56.866 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.863+0000 7ff484499700 1 -- start start 2022-11-27T16:14:56.869 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff484499700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff47c157b70 0x7ff47c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:56.869 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff484499700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c0c94f0 con 0x7ff47c15c1b0 2022-11-27T16:14:56.869 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff484499700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47c069570 con 0x7ff47c06db20 2022-11-27T16:14:56.870 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff484499700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47c069850 con 0x7ff47c157b70 2022-11-27T16:14:56.870 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff482235700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7ff47c157b70 0x7ff47c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:56.870 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff481a34700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7ff47c15c1b0 0x7ff47c0c8de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59336/0 (socket says 172.21.15.164:59336) 2022-11-27T16:14:56.870 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff481a34700 1 -- 172.21.15.164:0/401294415 learned_addr learned my addr 172.21.15.164:0/401294415 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:14:56.871 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff481233700 1 -- 172.21.15.164:0/401294415 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866135110 0 0) 0x7ff47c0c94f0 con 0x7ff47c15c1b0 2022-11-27T16:14:56.871 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff481233700 1 -- 172.21.15.164:0/401294415 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4640035b0 con 0x7ff47c15c1b0 2022-11-27T16:14:56.871 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff482235700 1 -- 172.21.15.164:0/401294415 >> v1:172.21.15.167:6789/0 conn(0x7ff47c06db20 legacy=0x7ff47c06df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:56.871 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff482235700 1 -- 172.21.15.164:0/401294415 >> v1:172.21.15.164:6789/0 conn(0x7ff47c15c1b0 legacy=0x7ff47c0c8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:56.872 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.864+0000 7ff482235700 1 -- 172.21.15.164:0/401294415 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c069b30 con 0x7ff47c157b70 2022-11-27T16:14:56.872 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff482235700 1 --2- 172.21.15.164:0/401294415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0cc970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009840 tx=0x7ff46c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1893c9e6bedcc1 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:56.872 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff481233700 1 -- 172.21.15.164:0/401294415 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011830 con 0x7ff47c157b70 2022-11-27T16:14:56.874 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff481233700 1 -- 172.21.15.164:0/401294415 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c011990 con 0x7ff47c157b70 2022-11-27T16:14:56.874 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff481233700 1 -- 172.21.15.164:0/401294415 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c011d20 con 0x7ff47c157b70 2022-11-27T16:14:56.875 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff484499700 1 -- 172.21.15.164:0/401294415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 msgr2=0x7ff47c0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:56.875 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff484499700 1 --2- 172.21.15.164:0/401294415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0cc970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff46c009840 tx=0x7ff46c009b50 comp rx=0 tx=0).stop 2022-11-27T16:14:56.876 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff484499700 1 -- 172.21.15.164:0/401294415 shutdown_connections 2022-11-27T16:14:56.876 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff484499700 1 --2- 172.21.15.164:0/401294415 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0cc970 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:56.877 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff484499700 1 -- 172.21.15.164:0/401294415 >> 172.21.15.164:0/401294415 conn(0x7ff47c0ce390 msgr2=0x7ff47c0ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:56.877 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff484499700 1 -- 172.21.15.164:0/401294415 shutdown_connections 2022-11-27T16:14:56.877 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.865+0000 7ff484499700 1 -- 172.21.15.164:0/401294415 wait complete. 2022-11-27T16:14:56.877 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 Processor -- start 2022-11-27T16:14:56.878 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 -- start start 2022-11-27T16:14:56.884 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff47c06db20 0x7ff47c0c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:56.884 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:56.885 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff47c15c1b0 0x7ff47c0c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:56.885 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c5460 con 0x7ff47c157b70 2022-11-27T16:14:56.886 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c6250 con 0x7ff47c15c1b0 2022-11-27T16:14:56.886 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff484499700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47c0c47f0 con 0x7ff47c06db20 2022-11-27T16:14:56.886 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff481a34700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:56.886 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff481a34700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0c66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36542/0 (socket says 172.21.15.164:36542) 2022-11-27T16:14:56.887 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.867+0000 7ff481a34700 1 -- 172.21.15.164:0/2641258289 learned_addr learned my addr 172.21.15.164:0/2641258289 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:14:56.887 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff482235700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff47c06db20 0x7ff47c0c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:56.887 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff482a36700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff47c15c1b0 0x7ff47c0c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:56.887 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff481a34700 1 -- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff47c06db20 msgr2=0x7ff47c0c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:56.888 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff481a34700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff47c06db20 0x7ff47c0c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:56.888 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff481a34700 1 -- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff47c15c1b0 msgr2=0x7ff47c0c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:56.888 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff481a34700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff47c15c1b0 0x7ff47c0c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:56.889 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff481a34700 1 -- 172.21.15.164:0/2641258289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c0094a0 con 0x7ff47c157b70 2022-11-27T16:14:56.889 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff482235700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff47c06db20 0x7ff47c0c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:14:56.889 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff481a34700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0c66e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c700 tx=0x7ff47800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:56.889 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff46affd700 1 -- 172.21.15.164:0/2641258289 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478005d70 con 0x7ff47c157b70 2022-11-27T16:14:56.890 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff46affd700 1 -- 172.21.15.164:0/2641258289 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4780070d0 con 0x7ff47c157b70 2022-11-27T16:14:56.890 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47c0c5640 con 0x7ff47c157b70 2022-11-27T16:14:56.890 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff46affd700 1 -- 172.21.15.164:0/2641258289 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478007460 con 0x7ff47c157b70 2022-11-27T16:14:56.890 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.868+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47c0c5c00 con 0x7ff47c157b70 2022-11-27T16:14:56.891 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.869+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff47c0c63c0 con 0x7ff47c157b70 2022-11-27T16:14:56.891 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.869+0000 7ff46affd700 1 -- 172.21.15.164:0/2641258289 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 4) v1 ==== 84457+0+0 (secure 0 0 0) 0x7ff478019030 con 0x7ff47c157b70 2022-11-27T16:14:56.891 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.870+0000 7ff46affd700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7ff45c066540 0x7ff45c0689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:14:56.891 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.870+0000 7ff46affd700 1 -- 172.21.15.164:0/2641258289 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7ff478081d40 con 0x7ff47c157b70 2022-11-27T16:14:56.892 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.870+0000 7ff482235700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7ff45c066540 0x7ff45c0689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:14:56.892 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.870+0000 7ff482235700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7ff45c066540 0x7ff45c0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff47c150a60 tx=0x7ff46c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:14:56.892 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:56.874+0000 7ff46affd700 1 -- 172.21.15.164:0/2641258289 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7ff478050310 con 0x7ff47c157b70 2022-11-27T16:14:57.055 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.052+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff47c072800 con 0x7ff47c157b70 2022-11-27T16:14:57.056 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.053+0000 7ff46affd700 1 -- 172.21.15.164:0/2641258289 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7011 (secure 0 0 0) 0x7ff478017020 con 0x7ff47c157b70 2022-11-27T16:14:57.056 INFO:teuthology.misc.health.smithi164.stdout: 2022-11-27T16:14:57.056 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":12,"fsid":"94c9dd9b-1805-4f9e-833a-3b29514744e3","created":"2022-11-27T16:14:46.055229+0000","modified":"2022-11-27T16:14:54.440720+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:14:48.616289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"fc87cc2c-4aba-41f8-9b94-8843abd34341","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2288745b-11fa-433b-bf88-dbfd0693401d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"82ed3e90-5d84-4fc3-ac13-a6a146bc60f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"864ca05d-4b9e-4111-8130-dd230c4a663c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"94560d98-a6fa-494f-8407-d432c650a7f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"815803f1-a95d-4921-a083-8185392357c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7e0d4655-210d-4601-8e1f-67a9fa7452ad","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"469ab427-7d6f-4f2a-9d32-275b76638d81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:14:57.058 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7ff45c066540 msgr2=0x7ff45c0689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:57.058 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7ff45c066540 0x7ff45c0689f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff47c150a60 tx=0x7ff46c008030 comp rx=0 tx=0).stop 2022-11-27T16:14:57.058 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 msgr2=0x7ff47c0c66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:14:57.058 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0c66e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff47800c700 tx=0x7ff47800cac0 comp rx=0 tx=0).stop 2022-11-27T16:14:57.059 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 shutdown_connections 2022-11-27T16:14:57.059 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7ff45c066540 0x7ff45c0689f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:57.059 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7ff47c15c1b0 0x7ff47c0c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:57.059 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7ff47c06db20 0x7ff47c0c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:57.060 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 --2- 172.21.15.164:0/2641258289 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7ff47c157b70 0x7ff47c0c66e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:14:57.060 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 >> 172.21.15.164:0/2641258289 conn(0x7ff47c0ce390 msgr2=0x7ff47c06a230 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:14:57.060 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 shutdown_connections 2022-11-27T16:14:57.060 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:14:57.055+0000 7ff484499700 1 -- 172.21.15.164:0/2641258289 wait complete. 2022-11-27T16:14:57.068 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T16:15:03.877 DEBUG:teuthology.orchestra.run.smithi164:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:15:03.999 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.997+0000 7f42dc495700 1 Processor -- start 2022-11-27T16:15:03.999 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.997+0000 7f42dc495700 1 -- start start 2022-11-27T16:15:04.002 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42dc495700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f42d40b1470 0x7f42d40cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.002 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42dc495700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d40cbd50 0x7f42d40cd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.003 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42dc495700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d40caf60 con 0x7f42d40b10a0 2022-11-27T16:15:04.003 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42dc495700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d4099510 con 0x7f42d40b1470 2022-11-27T16:15:04.003 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42dc495700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40cc2a0 con 0x7f42d40cbd50 2022-11-27T16:15:04.003 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42d9a30700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f42d40b1470 0x7f42d40cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.004 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42daa32700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d40cbd50 0x7f42d40cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.004 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42d9a30700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f42d40b1470 0x7f42d40cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.164:38002/0 (socket says 172.21.15.164:38002) 2022-11-27T16:15:04.004 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42d9a30700 1 -- 172.21.15.164:0/356330768 learned_addr learned my addr 172.21.15.164:0/356330768 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:04.004 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.998+0000 7f42da231700 1 --1- 172.21.15.164:0/356330768 >> v1:172.21.15.167:6789/0 conn(0x7f42d40b10a0 0x7f42d40b2bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.164:51658/0 (socket says 172.21.15.164:51658) 2022-11-27T16:15:04.005 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42daa32700 1 -- 172.21.15.164:0/356330768 >> v2:172.21.15.167:3300/0 conn(0x7f42d40b1470 msgr2=0x7f42d40cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:04.006 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42daa32700 1 --2- 172.21.15.164:0/356330768 >> v2:172.21.15.167:3300/0 conn(0x7f42d40b1470 0x7f42d40cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.006 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42daa32700 1 -- 172.21.15.164:0/356330768 >> v1:172.21.15.167:6789/0 conn(0x7f42d40b10a0 legacy=0x7f42d40b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.007 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42daa32700 1 -- 172.21.15.164:0/356330768 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d40cb200 con 0x7f42d40cbd50 2022-11-27T16:15:04.007 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42d9a30700 1 --2- 172.21.15.164:0/356330768 >> v2:172.21.15.167:3300/0 conn(0x7f42d40b1470 0x7f42d40cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:15:04.007 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42daa32700 1 --2- 172.21.15.164:0/356330768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d40cbd50 0x7f42d40cd930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009d90 tx=0x7f42cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a25b8ed5cb095e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:04.008 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42d922f700 1 -- 172.21.15.164:0/356330768 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00e070 con 0x7f42d40cbd50 2022-11-27T16:15:04.008 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42d922f700 1 -- 172.21.15.164:0/356330768 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc00bb20 con 0x7f42d40cbd50 2022-11-27T16:15:04.009 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:03.999+0000 7f42d922f700 1 -- 172.21.15.164:0/356330768 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc012430 con 0x7f42d40cbd50 2022-11-27T16:15:04.009 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 -- 172.21.15.164:0/356330768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d40cbd50 msgr2=0x7f42d40cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.010 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 --2- 172.21.15.164:0/356330768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d40cbd50 0x7f42d40cd930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009d90 tx=0x7f42cc00b330 comp rx=0 tx=0).stop 2022-11-27T16:15:04.010 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 -- 172.21.15.164:0/356330768 shutdown_connections 2022-11-27T16:15:04.011 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 --2- 172.21.15.164:0/356330768 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d40cbd50 0x7f42d40cd930 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.011 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 --2- 172.21.15.164:0/356330768 >> v2:172.21.15.167:3300/0 conn(0x7f42d40b1470 0x7f42d40cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.012 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 -- 172.21.15.164:0/356330768 >> 172.21.15.164:0/356330768 conn(0x7f42d4098fc0 msgr2=0x7f42d40cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:04.012 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 -- 172.21.15.164:0/356330768 shutdown_connections 2022-11-27T16:15:04.012 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.000+0000 7f42dc495700 1 -- 172.21.15.164:0/356330768 wait complete. 2022-11-27T16:15:04.013 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.001+0000 7f42dc495700 1 Processor -- start 2022-11-27T16:15:04.013 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.001+0000 7f42dc495700 1 -- start start 2022-11-27T16:15:04.020 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42dc495700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d40b10a0 0x7f42d40c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.020 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42dc495700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d40cbd50 0x7f42d40c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.021 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42dc495700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d414dd90 0x7f42d40c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.021 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42dc495700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d406b060 con 0x7f42d40cbd50 2022-11-27T16:15:04.022 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42dc495700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d406a4e0 con 0x7f42d40b10a0 2022-11-27T16:15:04.022 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42dc495700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d406a730 con 0x7f42d414dd90 2022-11-27T16:15:04.022 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d40cbd50 0x7f42d40c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.023 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d40cbd50 0x7f42d40c5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36770/0 (socket says 172.21.15.164:36770) 2022-11-27T16:15:04.023 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 -- 172.21.15.164:0/2018150259 learned_addr learned my addr 172.21.15.164:0/2018150259 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:04.024 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42da231700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d40b10a0 0x7f42d40c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.024 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42daa32700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d414dd90 0x7f42d40c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.025 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 -- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d414dd90 msgr2=0x7f42d40c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.025 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d414dd90 0x7f42d40c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.026 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 -- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d40b10a0 msgr2=0x7f42d40c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.026 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d40b10a0 0x7f42d40c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.027 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 -- 172.21.15.164:0/2018150259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d0009550 con 0x7f42d40cbd50 2022-11-27T16:15:04.027 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42d9a30700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d40cbd50 0x7f42d40c5f10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f42d000ebe0 tx=0x7f42d0009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:04.028 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42c2ffd700 1 -- 172.21.15.164:0/2018150259 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000c8b0 con 0x7f42d40cbd50 2022-11-27T16:15:04.028 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42c2ffd700 1 -- 172.21.15.164:0/2018150259 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000ca10 con 0x7f42d40cbd50 2022-11-27T16:15:04.029 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42cc009a70 con 0x7f42d40cbd50 2022-11-27T16:15:04.029 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.002+0000 7f42c2ffd700 1 -- 172.21.15.164:0/2018150259 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d000cdc0 con 0x7f42d40cbd50 2022-11-27T16:15:04.029 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.003+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d406acf0 con 0x7f42d40cbd50 2022-11-27T16:15:04.030 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.004+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d4072800 con 0x7f42d40cbd50 2022-11-27T16:15:04.030 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.004+0000 7f42c2ffd700 1 -- 172.21.15.164:0/2018150259 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f42d0019030 con 0x7f42d40cbd50 2022-11-27T16:15:04.031 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.004+0000 7f42c2ffd700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f42b4066590 0x7f42b4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.031 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.004+0000 7f42c2ffd700 1 -- 172.21.15.164:0/2018150259 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f42d0083370 con 0x7f42d40cbd50 2022-11-27T16:15:04.032 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.004+0000 7f42da231700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f42b4066590 0x7f42b4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.032 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.005+0000 7f42da231700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f42b4066590 0x7f42b4068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000fb0 tx=0x7f42c4007ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:04.033 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.009+0000 7f42c2ffd700 1 -- 172.21.15.164:0/2018150259 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f42d0050ed0 con 0x7f42d40cbd50 2022-11-27T16:15:04.191 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.190+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d40cb4e0 con 0x7f42d40cbd50 2022-11-27T16:15:04.192 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.190+0000 7f42c2ffd700 1 -- 172.21.15.164:0/2018150259 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12269 (secure 0 0 0) 0x7f42d00544f0 con 0x7f42d40cbd50 2022-11-27T16:15:04.192 INFO:teuthology.misc.health.smithi164.stdout: 2022-11-27T16:15:04.193 INFO:teuthology.misc.health.smithi164.stdout:{"epoch":16,"fsid":"94c9dd9b-1805-4f9e-833a-3b29514744e3","created":"2022-11-27T16:14:46.055229+0000","modified":"2022-11-27T16:15:00.304659+0000","last_up_change":"2022-11-27T16:14:57.299464+0000","last_in_change":"2022-11-27T16:14:48.616289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:14:57.480549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"fc87cc2c-4aba-41f8-9b94-8843abd34341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":101367},{"type":"v1","addr":"172.21.15.164:6801","nonce":101367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":101367},{"type":"v1","addr":"172.21.15.164:6803","nonce":101367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":101367},{"type":"v1","addr":"172.21.15.164:6807","nonce":101367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":101367},{"type":"v1","addr":"172.21.15.164:6805","nonce":101367}]},"public_addr":"172.21.15.164:6801/101367","cluster_addr":"172.21.15.164:6803/101367","heartbeat_back_addr":"172.21.15.164:6807/101367","heartbeat_front_addr":"172.21.15.164:6805/101367","state":["exists","up"]},{"osd":1,"uuid":"2288745b-11fa-433b-bf88-dbfd0693401d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":101368},{"type":"v1","addr":"172.21.15.164:6810","nonce":101368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":101368},{"type":"v1","addr":"172.21.15.164:6814","nonce":101368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":101368},{"type":"v1","addr":"172.21.15.164:6822","nonce":101368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":101368},{"type":"v1","addr":"172.21.15.164:6818","nonce":101368}]},"public_addr":"172.21.15.164:6810/101368","cluster_addr":"172.21.15.164:6814/101368","heartbeat_back_addr":"172.21.15.164:6822/101368","heartbeat_front_addr":"172.21.15.164:6818/101368","state":["exists","up"]},{"osd":2,"uuid":"82ed3e90-5d84-4fc3-ac13-a6a146bc60f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":101377},{"type":"v1","addr":"172.21.15.164:6825","nonce":101377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":101377},{"type":"v1","addr":"172.21.15.164:6827","nonce":101377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":101377},{"type":"v1","addr":"172.21.15.164:6831","nonce":101377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":101377},{"type":"v1","addr":"172.21.15.164:6829","nonce":101377}]},"public_addr":"172.21.15.164:6825/101377","cluster_addr":"172.21.15.164:6827/101377","heartbeat_back_addr":"172.21.15.164:6831/101377","heartbeat_front_addr":"172.21.15.164:6829/101377","state":["exists","up"]},{"osd":3,"uuid":"864ca05d-4b9e-4111-8130-dd230c4a663c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":101392},{"type":"v1","addr":"172.21.15.164:6811","nonce":101392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":101392},{"type":"v1","addr":"172.21.15.164:6815","nonce":101392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":101392},{"type":"v1","addr":"172.21.15.164:6823","nonce":101392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":101392},{"type":"v1","addr":"172.21.15.164:6819","nonce":101392}]},"public_addr":"172.21.15.164:6811/101392","cluster_addr":"172.21.15.164:6815/101392","heartbeat_back_addr":"172.21.15.164:6823/101392","heartbeat_front_addr":"172.21.15.164:6819/101392","state":["exists","up"]},{"osd":4,"uuid":"94560d98-a6fa-494f-8407-d432c650a7f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":101633},{"type":"v1","addr":"172.21.15.167:6809","nonce":101633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":101633},{"type":"v1","addr":"172.21.15.167:6811","nonce":101633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":101633},{"type":"v1","addr":"172.21.15.167:6815","nonce":101633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":101633},{"type":"v1","addr":"172.21.15.167:6813","nonce":101633}]},"public_addr":"172.21.15.167:6809/101633","cluster_addr":"172.21.15.167:6811/101633","heartbeat_back_addr":"172.21.15.167:6815/101633","heartbeat_front_addr":"172.21.15.167:6813/101633","state":["exists","up"]},{"osd":5,"uuid":"815803f1-a95d-4921-a083-8185392357c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6817","nonce":101647},{"type":"v1","addr":"172.21.15.167:6819","nonce":101647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":101647},{"type":"v1","addr":"172.21.15.167:6822","nonce":101647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":101647},{"type":"v1","addr":"172.21.15.167:6829","nonce":101647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":101647},{"type":"v1","addr":"172.21.15.167:6825","nonce":101647}]},"public_addr":"172.21.15.167:6819/101647","cluster_addr":"172.21.15.167:6822/101647","heartbeat_back_addr":"172.21.15.167:6829/101647","heartbeat_front_addr":"172.21.15.167:6825/101647","state":["exists","up"]},{"osd":6,"uuid":"7e0d4655-210d-4601-8e1f-67a9fa7452ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":101652},{"type":"v1","addr":"172.21.15.167:6818","nonce":101652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":101652},{"type":"v1","addr":"172.21.15.167:6823","nonce":101652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":101652},{"type":"v1","addr":"172.21.15.167:6831","nonce":101652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":101652},{"type":"v1","addr":"172.21.15.167:6828","nonce":101652}]},"public_addr":"172.21.15.167:6818/101652","cluster_addr":"172.21.15.167:6823/101652","heartbeat_back_addr":"172.21.15.167:6831/101652","heartbeat_front_addr":"172.21.15.167:6828/101652","state":["exists","up"]},{"osd":7,"uuid":"469ab427-7d6f-4f2a-9d32-275b76638d81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":101634},{"type":"v1","addr":"172.21.15.167:6801","nonce":101634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":101634},{"type":"v1","addr":"172.21.15.167:6803","nonce":101634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":101634},{"type":"v1","addr":"172.21.15.167:6807","nonce":101634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":101634},{"type":"v1","addr":"172.21.15.167:6805","nonce":101634}]},"public_addr":"172.21.15.167:6801/101634","cluster_addr":"172.21.15.167:6803/101634","heartbeat_back_addr":"172.21.15.167:6807/101634","heartbeat_front_addr":"172.21.15.167:6805/101634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.591828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.746590+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.798994+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.821213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.864723+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.826163+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.784349+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.812065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:15:04.196 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f42b4066590 msgr2=0x7f42b4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.196 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f42b4066590 0x7f42b4068a40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42c4000fb0 tx=0x7f42c4007ce0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.196 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d40cbd50 msgr2=0x7f42d40c5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.197 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d40cbd50 0x7f42d40c5f10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f42d000ebe0 tx=0x7f42d0009e50 comp rx=0 tx=0).stop 2022-11-27T16:15:04.197 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 shutdown_connections 2022-11-27T16:15:04.197 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f42b4066590 0x7f42b4068a40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.197 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f42d414dd90 0x7f42d40c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.198 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f42d40b10a0 0x7f42d40c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.198 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 --2- 172.21.15.164:0/2018150259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f42d40cbd50 0x7f42d40c5f10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.198 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 >> 172.21.15.164:0/2018150259 conn(0x7f42d4098fc0 msgr2=0x7f42d41591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:04.198 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 shutdown_connections 2022-11-27T16:15:04.198 INFO:teuthology.misc.health.smithi164.stderr:2022-11-27T16:15:04.193+0000 7f42dc495700 1 -- 172.21.15.164:0/2018150259 wait complete. 2022-11-27T16:15:04.205 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-11-27T16:15:04.206 INFO:tasks.ceph:Creating RBD pool 2022-11-27T16:15:04.206 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T16:15:04.325 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.324+0000 7fdc01f98700 1 Processor -- start 2022-11-27T16:15:04.326 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.324+0000 7fdc01f98700 1 -- start start 2022-11-27T16:15:04.340 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdc01f98700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fdbfc15fd50 0x7fdbfc070470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdc01f98700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc1597e0 con 0x7fdbfc159410 2022-11-27T16:15:04.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdc01f98700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbfc15bf90 con 0x7fdbfc15c470 2022-11-27T16:15:04.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdc01f98700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbfc1601f0 con 0x7fdbfc15fd50 2022-11-27T16:15:04.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfb7fe700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7fdbfc159410 0x7fdbfc0ca420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59576/0 (socket says 172.21.15.164:59576) 2022-11-27T16:15:04.342 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfb7fe700 1 -- 172.21.15.164:0/4217210072 learned_addr learned my addr 172.21.15.164:0/4217210072 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:15:04.342 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfaffd700 1 --2- 172.21.15.164:0/4217210072 >> v2:172.21.15.167:3300/0 conn(0x7fdbfc15fd50 0x7fdbfc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.342 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276364160 0 0) 0x7fdbfc1597e0 con 0x7fdbfc159410 2022-11-27T16:15:04.342 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbdc0035b0 con 0x7fdbfc159410 2022-11-27T16:15:04.343 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 922582034 0 0) 0x7fdbdc0035b0 con 0x7fdbfc159410 2022-11-27T16:15:04.343 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 >> v2:172.21.15.167:3300/0 conn(0x7fdbfc15fd50 msgr2=0x7fdbfc070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:04.343 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 --2- 172.21.15.164:0/4217210072 >> v2:172.21.15.167:3300/0 conn(0x7fdbfc15fd50 0x7fdbfc070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.344 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 >> v1:172.21.15.167:6789/0 conn(0x7fdbfc15c470 legacy=0x7fdbfc0cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.344 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0ce170 con 0x7fdbfc159410 2022-11-27T16:15:04.345 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 699751603 0 0) 0x7fdbec001670 con 0x7fdbfc159410 2022-11-27T16:15:04.345 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 >> v1:172.21.15.164:6789/0 conn(0x7fdbfc159410 legacy=0x7fdbfc0ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.346 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.325+0000 7fdbfa7fc700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbdc003a00 0x7fdbdc003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.346 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc004410 0x7fdbdc0048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.347 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc005170 0x7fdbdc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.347 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1601f0 con 0x7fdbdc004410 2022-11-27T16:15:04.348 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc00c0b0 con 0x7fdbdc003a00 2022-11-27T16:15:04.348 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc00c270 con 0x7fdbdc005170 2022-11-27T16:15:04.349 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 --2- 172.21.15.164:0/4217210072 >> v2:172.21.15.167:3300/0 conn(0x7fdbfc15fd50 0x7fdbfc070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:15:04.349 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc004410 0x7fdbdc0048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.350 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfb7fe700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbdc003a00 0x7fdbdc003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.350 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfbfff700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc005170 0x7fdbdc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.351 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 -- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc005170 msgr2=0x7fdbdc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:04.351 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc005170 0x7fdbdc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.351 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 -- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbdc003a00 msgr2=0x7fdbdc003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:04.352 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbdc003a00 0x7fdbdc003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.352 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 -- 172.21.15.164:0/4217210072 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc0035b0 con 0x7fdbdc004410 2022-11-27T16:15:04.352 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfbfff700 1 -- 172.21.15.164:0/4217210072 reap_dead start 2022-11-27T16:15:04.352 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfaffd700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc004410 0x7fdbdc0048d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d10 tx=0x7fdbdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53e684133661c7a server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:04.353 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe4009da0 con 0x7fdbdc004410 2022-11-27T16:15:04.353 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe400e9a0 con 0x7fdbdc004410 2022-11-27T16:15:04.353 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.326+0000 7fdbfa7fc700 1 -- 172.21.15.164:0/4217210072 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe400ed10 con 0x7fdbdc004410 2022-11-27T16:15:04.353 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.327+0000 7fdc01f98700 1 -- 172.21.15.164:0/4217210072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc004410 msgr2=0x7fdbdc0048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.353 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.327+0000 7fdc01f98700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc004410 0x7fdbdc0048d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4009d10 tx=0x7fdbdc005880 comp rx=0 tx=0).stop 2022-11-27T16:15:04.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.327+0000 7fdc01f98700 1 -- 172.21.15.164:0/4217210072 shutdown_connections 2022-11-27T16:15:04.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.327+0000 7fdc01f98700 1 --2- 172.21.15.164:0/4217210072 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbdc004410 0x7fdbdc0048d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.327+0000 7fdc01f98700 1 -- 172.21.15.164:0/4217210072 >> 172.21.15.164:0/4217210072 conn(0x7fdbfc06df40 msgr2=0x7fdbfc06e310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:04.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.327+0000 7fdc01f98700 1 -- 172.21.15.164:0/4217210072 shutdown_connections 2022-11-27T16:15:04.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.327+0000 7fdc01f98700 1 -- 172.21.15.164:0/4217210072 wait complete. 2022-11-27T16:15:04.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.328+0000 7fdc01f98700 1 Processor -- start 2022-11-27T16:15:04.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.328+0000 7fdc01f98700 1 -- start start 2022-11-27T16:15:04.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdc01f98700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc004410 0x7fdbfc1f0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.356 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdc01f98700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbfc0c78b0 0x7fdbfc15f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.356 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdc01f98700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1f4600 0x7fdbfc1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.356 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdc01f98700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1f4fe0 con 0x7fdbfc1f4600 2022-11-27T16:15:04.356 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdc01f98700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc1f0f40 con 0x7fdbfc0c78b0 2022-11-27T16:15:04.357 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdc01f98700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbfc0c7d20 con 0x7fdbdc004410 2022-11-27T16:15:04.357 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1f4600 0x7fdbfc1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.357 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1f4600 0x7fdbfc1f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36792/0 (socket says 172.21.15.164:36792) 2022-11-27T16:15:04.357 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 -- 172.21.15.164:0/2422391888 learned_addr learned my addr 172.21.15.164:0/2422391888 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:04.357 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfb7fe700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc004410 0x7fdbfc1f0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.358 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfaffd700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbfc0c78b0 0x7fdbfc15f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.358 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 -- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc004410 msgr2=0x7fdbfc1f0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.358 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc004410 0x7fdbfc1f0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.358 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 -- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbfc0c78b0 msgr2=0x7fdbfc15f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:04.359 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbfc0c78b0 0x7fdbfc15f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:04.359 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 -- 172.21.15.164:0/2422391888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0c8000 con 0x7fdbfc1f4600 2022-11-27T16:15:04.359 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.329+0000 7fdbfbfff700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1f4600 0x7fdbfc1f4aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ce8b0 tx=0x7fdbf400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:04.359 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.330+0000 7fdbf3fff700 1 -- 172.21.15.164:0/2422391888 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4003c10 con 0x7fdbfc1f4600 2022-11-27T16:15:04.360 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.330+0000 7fdbf3fff700 1 -- 172.21.15.164:0/2422391888 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf4003d70 con 0x7fdbfc1f4600 2022-11-27T16:15:04.360 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.330+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbfc0b3ed0 con 0x7fdbfc1f4600 2022-11-27T16:15:04.360 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.330+0000 7fdbf3fff700 1 -- 172.21.15.164:0/2422391888 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4005310 con 0x7fdbfc1f4600 2022-11-27T16:15:04.360 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.330+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbfc0b3b80 con 0x7fdbfc1f4600 2022-11-27T16:15:04.361 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.331+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbfc0b43f0 con 0x7fdbfc1f4600 2022-11-27T16:15:04.361 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.331+0000 7fdbf3fff700 1 -- 172.21.15.164:0/2422391888 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fdbf4005470 con 0x7fdbfc1f4600 2022-11-27T16:15:04.361 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.331+0000 7fdbf3fff700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fdbe0066590 0x7fdbe0068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:04.361 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.331+0000 7fdbf3fff700 1 -- 172.21.15.164:0/2422391888 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdbf4083480 con 0x7fdbfc1f4600 2022-11-27T16:15:04.361 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.331+0000 7fdbfb7fe700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fdbe0066590 0x7fdbe0068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:04.362 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.332+0000 7fdbfb7fe700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fdbe0066590 0x7fdbe0068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbec001eb0 tx=0x7fdbec001020 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:04.362 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.336+0000 7fdbf3fff700 1 -- 172.21.15.164:0/2422391888 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fdbfc0b43f0 con 0x7fdbfc1f4600 2022-11-27T16:15:04.517 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:04.516+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fdbfc0b43f0 con 0x7fdbfc1f4600 2022-11-27T16:15:05.331 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.329+0000 7fdbf3fff700 1 -- 172.21.15.164:0/2422391888 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fdbfc0b43f0 con 0x7fdbfc1f4600 2022-11-27T16:15:05.334 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fdbe0066590 msgr2=0x7fdbe0068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:05.335 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fdbe0066590 0x7fdbe0068a40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbec001eb0 tx=0x7fdbec001020 comp rx=0 tx=0).stop 2022-11-27T16:15:05.335 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1f4600 msgr2=0x7fdbfc1f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:05.335 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1f4600 0x7fdbfc1f4aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ce8b0 tx=0x7fdbf400ee50 comp rx=0 tx=0).stop 2022-11-27T16:15:05.336 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 shutdown_connections 2022-11-27T16:15:05.336 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fdbe0066590 0x7fdbe0068a40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.336 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fdbfc1f4600 0x7fdbfc1f4aa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.336 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fdbdc004410 0x7fdbfc1f0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.336 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 --2- 172.21.15.164:0/2422391888 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fdbfc0c78b0 0x7fdbfc15f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.337 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 >> 172.21.15.164:0/2422391888 conn(0x7fdbfc06df40 msgr2=0x7fdbfc15aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:05.337 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 shutdown_connections 2022-11-27T16:15:05.337 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.333+0000 7fdc01f98700 1 -- 172.21.15.164:0/2422391888 wait complete. 2022-11-27T16:15:05.338 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'rbd' created 2022-11-27T16:15:05.349 DEBUG:teuthology.orchestra.run.smithi164:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T16:15:05.486 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.475+0000 7f0216559700 1 Processor -- start 2022-11-27T16:15:05.486 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.475+0000 7f0216559700 1 -- start start 2022-11-27T16:15:05.487 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f0216559700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f0210159620 0x7f02100ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.487 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f0216559700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0210070560 con 0x7f0210159250 2022-11-27T16:15:05.487 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f0216559700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0210070840 con 0x7f021006aa90 2022-11-27T16:15:05.487 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f0216559700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02100ccbb0 con 0x7f0210159620 2022-11-27T16:15:05.488 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f020ffff700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f0210159250 0x7f021015f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59596/0 (socket says 172.21.15.164:59596) 2022-11-27T16:15:05.488 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f020ffff700 1 -- 172.21.15.164:0/3180728947 learned_addr learned my addr 172.21.15.164:0/3180728947 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:15:05.488 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f020f7fe700 1 --2- 172.21.15.164:0/3180728947 >> v2:172.21.15.167:3300/0 conn(0x7f0210159620 0x7f02100ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.488 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040554391 0 0) 0x7f0210070560 con 0x7f0210159250 2022-11-27T16:15:05.489 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.476+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 --> v1:172.21.15.164:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01fc0035b0 con 0x7f0210159250 2022-11-27T16:15:05.489 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 <== mon.0 v1:172.21.15.164:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1718455832 0 0) 0x7f01fc0035b0 con 0x7f0210159250 2022-11-27T16:15:05.489 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 >> v2:172.21.15.167:3300/0 conn(0x7f0210159620 msgr2=0x7f02100ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:05.489 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 --2- 172.21.15.164:0/3180728947 >> v2:172.21.15.167:3300/0 conn(0x7f0210159620 0x7f02100ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.489 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 >> v1:172.21.15.167:6789/0 conn(0x7f021006aa90 legacy=0x7f021006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:05.490 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 --> v1:172.21.15.164:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0210070b20 con 0x7f0210159250 2022-11-27T16:15:05.490 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 <== mon.0 v1:172.21.15.164:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 699751603 0 0) 0x7f0204001670 con 0x7f0210159250 2022-11-27T16:15:05.490 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 >> v1:172.21.15.164:6789/0 conn(0x7f0210159250 legacy=0x7f021015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:05.490 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01fc003a00 0x7f01fc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.490 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01fc004850 0x7f01fc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.491 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f01fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.491 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02100ccbb0 con 0x7f01fc005200 2022-11-27T16:15:05.491 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc00c110 con 0x7f01fc003a00 2022-11-27T16:15:05.491 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc00c2d0 con 0x7f01fc004850 2022-11-27T16:15:05.492 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f01fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.492 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020ffff700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01fc003a00 0x7f01fc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.492 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020f7fe700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01fc004850 0x7f01fc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.492 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 -- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01fc004850 msgr2=0x7f01fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:05.492 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f01fc004850 0x7f01fc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.493 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 -- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01fc003a00 msgr2=0x7f01fc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:05.493 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f01fc003a00 0x7f01fc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.493 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 -- 172.21.15.164:0/3180728947 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc0035b0 con 0x7f01fc005200 2022-11-27T16:15:05.493 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 -- 172.21.15.164:0/3180728947 reap_dead start 2022-11-27T16:15:05.493 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f0214af6700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f01fc00ba10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f020800c160 tx=0x7f01fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3083a0b9e14c58d4 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:05.494 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208001db0 con 0x7f01fc005200 2022-11-27T16:15:05.494 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020800ccc0 con 0x7f01fc005200 2022-11-27T16:15:05.494 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.477+0000 7f020effd700 1 -- 172.21.15.164:0/3180728947 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208003c40 con 0x7f01fc005200 2022-11-27T16:15:05.494 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.478+0000 7f0216559700 1 -- 172.21.15.164:0/3180728947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 msgr2=0x7f01fc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:05.495 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.478+0000 7f0216559700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f01fc00ba10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f020800c160 tx=0x7f01fc005880 comp rx=0 tx=0).stop 2022-11-27T16:15:05.495 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.478+0000 7f0216559700 1 -- 172.21.15.164:0/3180728947 shutdown_connections 2022-11-27T16:15:05.495 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.478+0000 7f0216559700 1 --2- 172.21.15.164:0/3180728947 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f01fc00ba10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.495 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.478+0000 7f0216559700 1 -- 172.21.15.164:0/3180728947 >> 172.21.15.164:0/3180728947 conn(0x7f02100cbb20 msgr2=0x7f02100ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:05.495 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.478+0000 7f0216559700 1 -- 172.21.15.164:0/3180728947 shutdown_connections 2022-11-27T16:15:05.496 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.478+0000 7f0216559700 1 -- 172.21.15.164:0/3180728947 wait complete. 2022-11-27T16:15:05.496 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.479+0000 7f0216559700 1 Processor -- start 2022-11-27T16:15:05.496 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.479+0000 7f0216559700 1 -- start start 2022-11-27T16:15:05.496 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f0216559700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f02100b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f0216559700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02100b3e50 0x7f02100ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f0216559700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02100abe70 0x7f02100af820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f0216559700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02100ae330 con 0x7f01fc005200 2022-11-27T16:15:05.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f0216559700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02100b1640 con 0x7f02100abe70 2022-11-27T16:15:05.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f0216559700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02100b1920 con 0x7f02100b3e50 2022-11-27T16:15:05.498 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f020ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f02100b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.498 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f020ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f02100b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36808/0 (socket says 172.21.15.164:36808) 2022-11-27T16:15:05.498 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f020ffff700 1 -- 172.21.15.164:0/3320413028 learned_addr learned my addr 172.21.15.164:0/3320413028 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:05.498 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.480+0000 7f0214af6700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02100abe70 0x7f02100af820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.499 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020f7fe700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02100b3e50 0x7f02100ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.499 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020ffff700 1 -- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02100b3e50 msgr2=0x7f02100ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:05.499 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020ffff700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02100b3e50 0x7f02100ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.499 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020ffff700 1 -- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02100abe70 msgr2=0x7f02100af820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:05.499 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020ffff700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02100abe70 0x7f02100af820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:05.500 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020ffff700 1 -- 172.21.15.164:0/3320413028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02100add70 con 0x7f01fc005200 2022-11-27T16:15:05.500 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020f7fe700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02100b3e50 0x7f02100ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:15:05.500 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020ffff700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f02100b2b90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02040072c0 tx=0x7f0204007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:05.500 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020cff9700 1 -- 172.21.15.164:0/3320413028 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040077a0 con 0x7f01fc005200 2022-11-27T16:15:05.501 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020cff9700 1 -- 172.21.15.164:0/3320413028 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020400f8d0 con 0x7f01fc005200 2022-11-27T16:15:05.501 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02100ae050 con 0x7f01fc005200 2022-11-27T16:15:05.501 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f020cff9700 1 -- 172.21.15.164:0/3320413028 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020400fc80 con 0x7f01fc005200 2022-11-27T16:15:05.501 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.481+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02100b30d0 con 0x7f01fc005200 2022-11-27T16:15:05.501 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.482+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021015f1d0 con 0x7f01fc005200 2022-11-27T16:15:05.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.482+0000 7f020cff9700 1 -- 172.21.15.164:0/3320413028 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f020401d460 con 0x7f01fc005200 2022-11-27T16:15:05.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.483+0000 7f020cff9700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f01f0066590 0x7f01f0068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:05.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.483+0000 7f020cff9700 1 -- 172.21.15.164:0/3320413028 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f02040835b0 con 0x7f01fc005200 2022-11-27T16:15:05.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.483+0000 7f020f7fe700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f01f0066590 0x7f01f0068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:05.503 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.483+0000 7f020f7fe700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f01f0066590 0x7f01f0068a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0200009ae0 tx=0x7f0200008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:05.503 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.487+0000 7f020cff9700 1 -- 172.21.15.164:0/3320413028 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f0204051070 con 0x7f01fc005200 2022-11-27T16:15:05.672 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:05.671+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f02100b32c0 con 0x7f01fc005200 2022-11-27T16:15:06.336 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.335+0000 7f020cff9700 1 -- 172.21.15.164:0/3320413028 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f02100b32c0 con 0x7f01fc005200 2022-11-27T16:15:06.339 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f01f0066590 msgr2=0x7f01f0068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.340 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f01f0066590 0x7f01f0068a40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0200009ae0 tx=0x7f0200008040 comp rx=0 tx=0).stop 2022-11-27T16:15:06.340 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 msgr2=0x7f02100b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.340 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f02100b2b90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f02040072c0 tx=0x7f0204007680 comp rx=0 tx=0).stop 2022-11-27T16:15:06.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 shutdown_connections 2022-11-27T16:15:06.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f01f0066590 0x7f01f0068a40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f02100abe70 0x7f02100af820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f01fc005200 0x7f02100b2b90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.341 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 --2- 172.21.15.164:0/3320413028 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f02100b3e50 0x7f02100ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.342 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 >> 172.21.15.164:0/3320413028 conn(0x7f02100cbb20 msgr2=0x7f02100ce810 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:06.342 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 shutdown_connections 2022-11-27T16:15:06.342 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.338+0000 7f0216559700 1 -- 172.21.15.164:0/3320413028 wait complete. 2022-11-27T16:15:06.343 INFO:teuthology.orchestra.run.smithi164.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T16:15:06.355 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T16:15:06.356 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T16:15:06.356 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T16:15:06.359 INFO:tasks.ceph.mds.a:Started 2022-11-27T16:15:06.359 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T16:15:06.360 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T16:15:06.362 INFO:tasks.ceph.mds.c:Started 2022-11-27T16:15:06.362 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T16:15:06.363 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T16:15:06.365 INFO:tasks.ceph.mds.b:Started 2022-11-27T16:15:06.366 INFO:tasks.ceph.mds.d:Restarting daemon 2022-11-27T16:15:06.366 DEBUG:teuthology.orchestra.run.smithi167:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-11-27T16:15:06.368 INFO:tasks.ceph.mds.d:Started 2022-11-27T16:15:06.369 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T16:15:06.369 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:15:06.444 INFO:tasks.ceph.mds.a.smithi164.stdout:starting mds.a at 2022-11-27T16:15:06.444 INFO:tasks.ceph.mds.c.smithi164.stdout:starting mds.c at 2022-11-27T16:15:06.445 INFO:tasks.ceph.mds.c.smithi164.stderr:2022-11-27T16:15:06.443+0000 7f9412ac6700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2022-11-27T16:15:06.450 INFO:tasks.ceph.mds.b.smithi167.stdout:starting mds.b at 2022-11-27T16:15:06.450 INFO:tasks.ceph.mds.d.smithi167.stdout:starting mds.d at 2022-11-27T16:15:06.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.500+0000 7f2f9a9be700 1 Processor -- start 2022-11-27T16:15:06.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.500+0000 7f2f9a9be700 1 -- start start 2022-11-27T16:15:06.510 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.501+0000 7f2f9a9be700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f940b75e0 0x7f2f9415f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.510 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.501+0000 7f2f9a9be700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f940ccba0 con 0x7f2f940cf5f0 2022-11-27T16:15:06.511 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.501+0000 7f2f9a9be700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f940d0750 con 0x7f2f940d1910 2022-11-27T16:15:06.511 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.501+0000 7f2f9a9be700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f940cc620 con 0x7f2f940b75e0 2022-11-27T16:15:06.511 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.501+0000 7f2f93fff700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f2f940b75e0 0x7f2f9415f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.511 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.501+0000 7f2f937fe700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f2f940cf5f0 0x7f2f9415f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59638/0 (socket says 172.21.15.164:59638) 2022-11-27T16:15:06.512 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.501+0000 7f2f937fe700 1 -- 172.21.15.164:0/2248792319 learned_addr learned my addr 172.21.15.164:0/2248792319 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:15:06.512 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f93fff700 1 -- 172.21.15.164:0/2248792319 >> v1:172.21.15.167:6789/0 conn(0x7f2f940d1910 legacy=0x7f2f940d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.512 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f93fff700 1 -- 172.21.15.164:0/2248792319 >> v1:172.21.15.164:6789/0 conn(0x7f2f940cf5f0 legacy=0x7f2f9415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.512 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f92ffd700 1 -- 172.21.15.164:0/2248792319 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393373220 0 0) 0x7f2f940ccba0 con 0x7f2f940cf5f0 2022-11-27T16:15:06.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f93fff700 1 -- 172.21.15.164:0/2248792319 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f940d0a30 con 0x7f2f940b75e0 2022-11-27T16:15:06.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f93fff700 1 --2- 172.21.15.164:0/2248792319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940b75e0 0x7f2f9415f370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009840 tx=0x7f2f88009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fea66adec5469b server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:06.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f92ffd700 1 -- 172.21.15.164:0/2248792319 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88011830 con 0x7f2f940b75e0 2022-11-27T16:15:06.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f92ffd700 1 -- 172.21.15.164:0/2248792319 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f88011990 con 0x7f2f940b75e0 2022-11-27T16:15:06.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f92ffd700 1 -- 172.21.15.164:0/2248792319 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f88011d20 con 0x7f2f940b75e0 2022-11-27T16:15:06.514 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2248792319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940b75e0 msgr2=0x7f2f9415f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.514 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2248792319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940b75e0 0x7f2f9415f370 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009840 tx=0x7f2f88009b50 comp rx=0 tx=0).stop 2022-11-27T16:15:06.514 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.502+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2248792319 shutdown_connections 2022-11-27T16:15:06.515 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.503+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2248792319 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940b75e0 0x7f2f9415f370 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.515 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.503+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2248792319 >> 172.21.15.164:0/2248792319 conn(0x7f2f94070b00 msgr2=0x7f2f94070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:06.515 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.503+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2248792319 shutdown_connections 2022-11-27T16:15:06.516 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.503+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2248792319 wait complete. 2022-11-27T16:15:06.516 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.504+0000 7f2f9a9be700 1 Processor -- start 2022-11-27T16:15:06.517 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.504+0000 7f2f9a9be700 1 -- start start 2022-11-27T16:15:06.517 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f940b75e0 0x7f2f940a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.518 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940cf5f0 0x7f2f940b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.518 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940d1910 0x7f2f940b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.519 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940b3bd0 con 0x7f2f940cf5f0 2022-11-27T16:15:06.519 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940b3090 con 0x7f2f940d1910 2022-11-27T16:15:06.520 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f940b3330 con 0x7f2f940b75e0 2022-11-27T16:15:06.520 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940cf5f0 0x7f2f940b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.521 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940cf5f0 0x7f2f940b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36844/0 (socket says 172.21.15.164:36844) 2022-11-27T16:15:06.521 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 -- 172.21.15.164:0/2607214828 learned_addr learned my addr 172.21.15.164:0/2607214828 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:06.521 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f98f5b700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940d1910 0x7f2f940b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.522 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f93fff700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f940b75e0 0x7f2f940a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.522 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 -- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f940b75e0 msgr2=0x7f2f940a96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.523 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f940b75e0 0x7f2f940a96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.523 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 -- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940d1910 msgr2=0x7f2f940b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.524 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940d1910 0x7f2f940b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.524 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 -- 172.21.15.164:0/2607214828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f880094a0 con 0x7f2f940cf5f0 2022-11-27T16:15:06.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f937fe700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940cf5f0 0x7f2f940b2b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400c520 tx=0x7f2f8400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:06.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f90ff9700 1 -- 172.21.15.164:0/2607214828 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84004c80 con 0x7f2f940cf5f0 2022-11-27T16:15:06.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f90ff9700 1 -- 172.21.15.164:0/2607214828 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f84004e00 con 0x7f2f940cf5f0 2022-11-27T16:15:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f940b3610 con 0x7f2f940cf5f0 2022-11-27T16:15:06.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f90ff9700 1 -- 172.21.15.164:0/2607214828 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84007310 con 0x7f2f940cf5f0 2022-11-27T16:15:06.527 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.505+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f940c6de0 con 0x7f2f940cf5f0 2022-11-27T16:15:06.527 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.506+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f940c6c40 con 0x7f2f940cf5f0 2022-11-27T16:15:06.527 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.507+0000 7f2f90ff9700 1 -- 172.21.15.164:0/2607214828 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f2f8401a020 con 0x7f2f940cf5f0 2022-11-27T16:15:06.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.507+0000 7f2f90ff9700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f2f74066590 0x7f2f74068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.507+0000 7f2f93fff700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f2f74066590 0x7f2f74068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.507+0000 7f2f90ff9700 1 -- 172.21.15.164:0/2607214828 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f2f840830d0 con 0x7f2f940cf5f0 2022-11-27T16:15:06.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.508+0000 7f2f93fff700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f2f74066590 0x7f2f74068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f880090b0 tx=0x7f2f8801c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:06.529 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.512+0000 7f2f90ff9700 1 -- 172.21.15.164:0/2607214828 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f2f940c6c40 con 0x7f2f940cf5f0 2022-11-27T16:15:06.696 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.694+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f940c6c40 con 0x7f2f940cf5f0 2022-11-27T16:15:06.697 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.695+0000 7f2f90ff9700 1 -- 172.21.15.164:0/2607214828 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13802 (secure 0 0 0) 0x7f2f940c6c40 con 0x7f2f940cf5f0 2022-11-27T16:15:06.697 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:15:06.697 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":18,"fsid":"94c9dd9b-1805-4f9e-833a-3b29514744e3","created":"2022-11-27T16:14:46.055229+0000","modified":"2022-11-27T16:15:06.328464+0000","last_up_change":"2022-11-27T16:14:57.299464+0000","last_in_change":"2022-11-27T16:14:48.616289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:14:57.480549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T16:15:04.534458+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"fc87cc2c-4aba-41f8-9b94-8843abd34341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":101367},{"type":"v1","addr":"172.21.15.164:6801","nonce":101367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":101367},{"type":"v1","addr":"172.21.15.164:6803","nonce":101367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":101367},{"type":"v1","addr":"172.21.15.164:6807","nonce":101367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":101367},{"type":"v1","addr":"172.21.15.164:6805","nonce":101367}]},"public_addr":"172.21.15.164:6801/101367","cluster_addr":"172.21.15.164:6803/101367","heartbeat_back_addr":"172.21.15.164:6807/101367","heartbeat_front_addr":"172.21.15.164:6805/101367","state":["exists","up"]},{"osd":1,"uuid":"2288745b-11fa-433b-bf88-dbfd0693401d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":101368},{"type":"v1","addr":"172.21.15.164:6810","nonce":101368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":101368},{"type":"v1","addr":"172.21.15.164:6814","nonce":101368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":101368},{"type":"v1","addr":"172.21.15.164:6822","nonce":101368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":101368},{"type":"v1","addr":"172.21.15.164:6818","nonce":101368}]},"public_addr":"172.21.15.164:6810/101368","cluster_addr":"172.21.15.164:6814/101368","heartbeat_back_addr":"172.21.15.164:6822/101368","heartbeat_front_addr":"172.21.15.164:6818/101368","state":["exists","up"]},{"osd":2,"uuid":"82ed3e90-5d84-4fc3-ac13-a6a146bc60f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":101377},{"type":"v1","addr":"172.21.15.164:6825","nonce":101377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":101377},{"type":"v1","addr":"172.21.15.164:6827","nonce":101377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":101377},{"type":"v1","addr":"172.21.15.164:6831","nonce":101377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":101377},{"type":"v1","addr":"172.21.15.164:6829","nonce":101377}]},"public_addr":"172.21.15.164:6825/101377","cluster_addr":"172.21.15.164:6827/101377","heartbeat_back_addr":"172.21.15.164:6831/101377","heartbeat_front_addr":"172.21.15.164:6829/101377","state":["exists","up"]},{"osd":3,"uuid":"864ca05d-4b9e-4111-8130-dd230c4a663c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":101392},{"type":"v1","addr":"172.21.15.164:6811","nonce":101392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":101392},{"type":"v1","addr":"172.21.15.164:6815","nonce":101392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":101392},{"type":"v1","addr":"172.21.15.164:6823","nonce":101392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":101392},{"type":"v1","addr":"172.21.15.164:6819","nonce":101392}]},"public_addr":"172.21.15.164:6811/101392","cluster_addr":"172.21.15.164:6815/101392","heartbeat_back_addr":"172.21.15.164:6823/101392","heartbeat_front_addr":"172.21.15.164:6819/101392","state":["exists","up"]},{"osd":4,"uuid":"94560d98-a6fa-494f-8407-d432c650a7f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":101633},{"type":"v1","addr":"172.21.15.167:6809","nonce":101633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":101633},{"type":"v1","addr":"172.21.15.167:6811","nonce":101633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":101633},{"type":"v1","addr":"172.21.15.167:6815","nonce":101633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":101633},{"type":"v1","addr":"172.21.15.167:6813","nonce":101633}]},"public_addr":"172.21.15.167:6809/101633","cluster_addr":"172.21.15.167:6811/101633","heartbeat_back_addr":"172.21.15.167:6815/101633","heartbeat_front_addr":"172.21.15.167:6813/101633","state":["exists","up"]},{"osd":5,"uuid":"815803f1-a95d-4921-a083-8185392357c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6817","nonce":101647},{"type":"v1","addr":"172.21.15.167:6819","nonce":101647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":101647},{"type":"v1","addr":"172.21.15.167:6822","nonce":101647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":101647},{"type":"v1","addr":"172.21.15.167:6829","nonce":101647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":101647},{"type":"v1","addr":"172.21.15.167:6825","nonce":101647}]},"public_addr":"172.21.15.167:6819/101647","cluster_addr":"172.21.15.167:6822/101647","heartbeat_back_addr":"172.21.15.167:6829/101647","heartbeat_front_addr":"172.21.15.167:6825/101647","state":["exists","up"]},{"osd":6,"uuid":"7e0d4655-210d-4601-8e1f-67a9fa7452ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":101652},{"type":"v1","addr":"172.21.15.167:6818","nonce":101652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":101652},{"type":"v1","addr":"172.21.15.167:6823","nonce":101652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":101652},{"type":"v1","addr":"172.21.15.167:6831","nonce":101652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":101652},{"type":"v1","addr":"172.21.15.167:6828","nonce":101652}]},"public_addr":"172.21.15.167:6818/101652","cluster_addr":"172.21.15.167:6823/101652","heartbeat_back_addr":"172.21.15.167:6831/101652","heartbeat_front_addr":"172.21.15.167:6828/101652","state":["exists","up"]},{"osd":7,"uuid":"469ab427-7d6f-4f2a-9d32-275b76638d81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":101634},{"type":"v1","addr":"172.21.15.167:6801","nonce":101634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":101634},{"type":"v1","addr":"172.21.15.167:6803","nonce":101634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":101634},{"type":"v1","addr":"172.21.15.167:6807","nonce":101634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":101634},{"type":"v1","addr":"172.21.15.167:6805","nonce":101634}]},"public_addr":"172.21.15.167:6801/101634","cluster_addr":"172.21.15.167:6803/101634","heartbeat_back_addr":"172.21.15.167:6807/101634","heartbeat_front_addr":"172.21.15.167:6805/101634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.591828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.746590+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.798994+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.821213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.864723+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.826163+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.784349+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.812065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:15:06.699 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f2f74066590 msgr2=0x7f2f74068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.699 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f2f74066590 0x7f2f74068a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2f880090b0 tx=0x7f2f8801c040 comp rx=0 tx=0).stop 2022-11-27T16:15:06.700 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940cf5f0 msgr2=0x7f2f940b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.700 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940cf5f0 0x7f2f940b2b50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400c520 tx=0x7f2f8400c8e0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.700 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 shutdown_connections 2022-11-27T16:15:06.700 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f2f74066590 0x7f2f74068a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.701 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2f940d1910 0x7f2f940b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.701 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2f940b75e0 0x7f2f940a96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.701 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 --2- 172.21.15.164:0/2607214828 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2f940cf5f0 0x7f2f940b2b50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.701 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.697+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 >> 172.21.15.164:0/2607214828 conn(0x7f2f94070b00 msgr2=0x7f2f940b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:06.702 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.698+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 shutdown_connections 2022-11-27T16:15:06.702 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.698+0000 7f2f9a9be700 1 -- 172.21.15.164:0/2607214828 wait complete. 2022-11-27T16:15:06.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:14:57.480549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:15:04.534458+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T16:15:06.712 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:15:06.837 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.835+0000 7f99d3b8a700 1 Processor -- start 2022-11-27T16:15:06.837 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.836+0000 7f99d3b8a700 1 -- start start 2022-11-27T16:15:06.840 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.836+0000 7f99d3b8a700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f99cc0ae0d0 0x7f99cc06e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.841 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.836+0000 7f99d3b8a700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc0b1f80 con 0x7f99cc0b3420 2022-11-27T16:15:06.841 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.836+0000 7f99d3b8a700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc1600f0 con 0x7f99cc0693d0 2022-11-27T16:15:06.842 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.836+0000 7f99d3b8a700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc06efa0 con 0x7f99cc0ae0d0 2022-11-27T16:15:06.842 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.836+0000 7f99d1926700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f99cc0ae0d0 0x7f99cc06e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.842 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d1926700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f99cc0ae0d0 0x7f99cc06e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36850/0 (socket says 172.21.15.164:36850) 2022-11-27T16:15:06.842 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d2127700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f99cc0b3420 0x7f99cc15f9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59652/0 (socket says 172.21.15.164:59652) 2022-11-27T16:15:06.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d1926700 1 -- 172.21.15.164:0/2853947703 learned_addr learned my addr 172.21.15.164:0/2853947703 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:06.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d1926700 1 -- 172.21.15.164:0/2853947703 >> v1:172.21.15.167:6789/0 conn(0x7f99cc0693d0 legacy=0x7f99cc15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d1926700 1 -- 172.21.15.164:0/2853947703 >> v1:172.21.15.164:6789/0 conn(0x7f99cc0b3420 legacy=0x7f99cc15f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.843 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d1926700 1 -- 172.21.15.164:0/2853947703 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc06f280 con 0x7f99cc0ae0d0 2022-11-27T16:15:06.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d0924700 1 -- 172.21.15.164:0/2853947703 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406553358 0 0) 0x7f99cc0b1f80 con 0x7f99cc0b3420 2022-11-27T16:15:06.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d1926700 1 --2- 172.21.15.164:0/2853947703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0ae0d0 0x7f99cc06e630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0098f0 tx=0x7f99bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f790cb2aa810c9 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:06.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d0924700 1 -- 172.21.15.164:0/2853947703 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc004230 con 0x7f99cc0ae0d0 2022-11-27T16:15:06.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d0924700 1 -- 172.21.15.164:0/2853947703 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc004390 con 0x7f99cc0ae0d0 2022-11-27T16:15:06.845 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.837+0000 7f99d0924700 1 -- 172.21.15.164:0/2853947703 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0036a0 con 0x7f99cc0ae0d0 2022-11-27T16:15:06.845 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.838+0000 7f99d3b8a700 1 -- 172.21.15.164:0/2853947703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0ae0d0 msgr2=0x7f99cc06e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.845 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.838+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/2853947703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0ae0d0 0x7f99cc06e630 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f99bc0098f0 tx=0x7f99bc009c00 comp rx=0 tx=0).stop 2022-11-27T16:15:06.845 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.838+0000 7f99d3b8a700 1 -- 172.21.15.164:0/2853947703 shutdown_connections 2022-11-27T16:15:06.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.838+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/2853947703 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0ae0d0 0x7f99cc06e630 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.838+0000 7f99d3b8a700 1 -- 172.21.15.164:0/2853947703 >> 172.21.15.164:0/2853947703 conn(0x7f99cc06ce40 msgr2=0x7f99cc06d240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:06.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.838+0000 7f99d3b8a700 1 -- 172.21.15.164:0/2853947703 shutdown_connections 2022-11-27T16:15:06.846 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.838+0000 7f99d3b8a700 1 -- 172.21.15.164:0/2853947703 wait complete. 2022-11-27T16:15:06.847 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.839+0000 7f99d3b8a700 1 Processor -- start 2022-11-27T16:15:06.847 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.839+0000 7f99d3b8a700 1 -- start start 2022-11-27T16:15:06.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99cc0693d0 0x7f99cc0abd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99cc0ae0d0 0x7f99cc15eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0b3420 0x7f99cc149cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15fe00 con 0x7f99cc0b3420 2022-11-27T16:15:06.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15f560 con 0x7f99cc0693d0 2022-11-27T16:15:06.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15f840 con 0x7f99cc0ae0d0 2022-11-27T16:15:06.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0b3420 0x7f99cc149cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0b3420 0x7f99cc149cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36858/0 (socket says 172.21.15.164:36858) 2022-11-27T16:15:06.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 -- 172.21.15.164:0/1209902691 learned_addr learned my addr 172.21.15.164:0/1209902691 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:06.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d1125700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99cc0ae0d0 0x7f99cc15eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d1926700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99cc0693d0 0x7f99cc0abd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 -- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99cc0ae0d0 msgr2=0x7f99cc15eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99cc0ae0d0 0x7f99cc15eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 -- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99cc0693d0 msgr2=0x7f99cc0abd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:06.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99cc0693d0 0x7f99cc0abd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:06.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 -- 172.21.15.164:0/1209902691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc009550 con 0x7f99cc0b3420 2022-11-27T16:15:06.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d1926700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99cc0693d0 0x7f99cc0abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:15:06.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d2127700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0b3420 0x7f99cc149cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f99c400cb60 tx=0x7f99c400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:06.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99ba7fc700 1 -- 172.21.15.164:0/1209902691 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400ab50 con 0x7f99cc0b3420 2022-11-27T16:15:06.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99ba7fc700 1 -- 172.21.15.164:0/1209902691 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c400acb0 con 0x7f99cc0b3420 2022-11-27T16:15:06.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc15fb20 con 0x7f99cc0b3420 2022-11-27T16:15:06.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99ba7fc700 1 -- 172.21.15.164:0/1209902691 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4007310 con 0x7f99cc0b3420 2022-11-27T16:15:06.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.840+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc15c300 con 0x7f99cc0b3420 2022-11-27T16:15:06.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.841+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc0c3c10 con 0x7f99cc0b3420 2022-11-27T16:15:06.858 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.842+0000 7f99ba7fc700 1 -- 172.21.15.164:0/1209902691 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f99c401a030 con 0x7f99cc0b3420 2022-11-27T16:15:06.858 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.842+0000 7f99ba7fc700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f99ac066590 0x7f99ac068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:06.858 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.842+0000 7f99ba7fc700 1 -- 172.21.15.164:0/1209902691 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f99c40830f0 con 0x7f99cc0b3420 2022-11-27T16:15:06.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.842+0000 7f99d1926700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f99ac066590 0x7f99ac068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:06.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.842+0000 7f99d1926700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f99ac066590 0x7f99ac068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009c40 tx=0x7f99bc008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:06.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:06.846+0000 7f99ba7fc700 1 -- 172.21.15.164:0/1209902691 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f99cc0c3c10 con 0x7f99cc0b3420 2022-11-27T16:15:07.024 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.022+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f99cc0c3c10 con 0x7f99cc0b3420 2022-11-27T16:15:07.024 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.023+0000 7f99ba7fc700 1 -- 172.21.15.164:0/1209902691 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f99cc0c3c10 con 0x7f99cc0b3420 2022-11-27T16:15:07.025 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2022-11-27T16:15:07.027 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f99ac066590 msgr2=0x7f99ac068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.027 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f99ac066590 0x7f99ac068a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009c40 tx=0x7f99bc008030 comp rx=0 tx=0).stop 2022-11-27T16:15:07.027 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0b3420 msgr2=0x7f99cc149cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.027 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0b3420 0x7f99cc149cd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f99c400cb60 tx=0x7f99c400cf20 comp rx=0 tx=0).stop 2022-11-27T16:15:07.028 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 shutdown_connections 2022-11-27T16:15:07.028 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f99ac066590 0x7f99ac068a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.028 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f99cc0b3420 0x7f99cc149cd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.028 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f99cc0693d0 0x7f99cc0abd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.029 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 --2- 172.21.15.164:0/1209902691 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f99cc0ae0d0 0x7f99cc15eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.029 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 >> 172.21.15.164:0/1209902691 conn(0x7f99cc06ce40 msgr2=0x7f99cc15ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:07.029 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 shutdown_connections 2022-11-27T16:15:07.029 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.025+0000 7f99d3b8a700 1 -- 172.21.15.164:0/1209902691 wait complete. 2022-11-27T16:15:07.038 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:15:07.160 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.159+0000 7f0776279700 1 Processor -- start 2022-11-27T16:15:07.161 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.159+0000 7f0776279700 1 -- start start 2022-11-27T16:15:07.164 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f0776279700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f07700ae0d0 0x7f077006e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.164 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f0776279700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07700693d0 0x7f0770069820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.164 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f0776279700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0770069d60 con 0x7f077015f2d0 2022-11-27T16:15:07.165 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f0776279700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07700b1f80 con 0x7f07700ae0d0 2022-11-27T16:15:07.165 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f0776279700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077006efa0 con 0x7f07700693d0 2022-11-27T16:15:07.165 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f076ffff700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f07700ae0d0 0x7f077006e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.166 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f076f7fe700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07700693d0 0x7f0770069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.166 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f0774816700 1 --1- >> v1:172.21.15.167:6789/0 conn(0x7f077015f2d0 0x7f077015f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.167:6789/0 says I am v1:172.21.15.164:51760/0 (socket says 172.21.15.164:51760) 2022-11-27T16:15:07.166 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f0774816700 1 -- 172.21.15.164:0/3011887659 learned_addr learned my addr 172.21.15.164:0/3011887659 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:15:07.166 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f076f7fe700 1 -- 172.21.15.164:0/3011887659 >> v2:172.21.15.167:3300/0 conn(0x7f07700ae0d0 msgr2=0x7f077006e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:07.166 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f076f7fe700 1 --2- 172.21.15.164:0/3011887659 >> v2:172.21.15.167:3300/0 conn(0x7f07700ae0d0 0x7f077006e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.167 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f076f7fe700 1 -- 172.21.15.164:0/3011887659 >> v1:172.21.15.167:6789/0 conn(0x7f077015f2d0 legacy=0x7f077015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.167 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f076f7fe700 1 -- 172.21.15.164:0/3011887659 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077006f280 con 0x7f07700693d0 2022-11-27T16:15:07.167 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.160+0000 7f076ffff700 1 --2- 172.21.15.164:0/3011887659 >> v2:172.21.15.167:3300/0 conn(0x7f07700ae0d0 0x7f077006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:15:07.167 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f076f7fe700 1 --2- 172.21.15.164:0/3011887659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07700693d0 0x7f0770069820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0758009840 tx=0x7f0758009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b93821c8438628a3 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.168 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f076effd700 1 -- 172.21.15.164:0/3011887659 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758011830 con 0x7f07700693d0 2022-11-27T16:15:07.168 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f076effd700 1 -- 172.21.15.164:0/3011887659 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0758011990 con 0x7f07700693d0 2022-11-27T16:15:07.169 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f076effd700 1 -- 172.21.15.164:0/3011887659 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758011d20 con 0x7f07700693d0 2022-11-27T16:15:07.169 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 -- 172.21.15.164:0/3011887659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07700693d0 msgr2=0x7f0770069820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.169 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 --2- 172.21.15.164:0/3011887659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07700693d0 0x7f0770069820 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0758009840 tx=0x7f0758009b50 comp rx=0 tx=0).stop 2022-11-27T16:15:07.170 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 -- 172.21.15.164:0/3011887659 shutdown_connections 2022-11-27T16:15:07.170 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 --2- 172.21.15.164:0/3011887659 >> v2:172.21.15.167:3300/0 conn(0x7f07700ae0d0 0x7f077006e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.170 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 --2- 172.21.15.164:0/3011887659 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f07700693d0 0x7f0770069820 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.170 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 -- 172.21.15.164:0/3011887659 >> 172.21.15.164:0/3011887659 conn(0x7f077006ce40 msgr2=0x7f077006d240 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:07.171 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 -- 172.21.15.164:0/3011887659 shutdown_connections 2022-11-27T16:15:07.171 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.161+0000 7f0776279700 1 -- 172.21.15.164:0/3011887659 wait complete. 2022-11-27T16:15:07.171 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 Processor -- start 2022-11-27T16:15:07.172 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 -- start start 2022-11-27T16:15:07.175 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07700693d0 0x7f0770158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07700ae0d0 0x7f07700b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f077015f2d0 0x7f07700b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07700c3400 con 0x7f07700693d0 2022-11-27T16:15:07.176 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077015bf30 con 0x7f07700ae0d0 2022-11-27T16:15:07.177 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.163+0000 7f0776279700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07700c2830 con 0x7f077015f2d0 2022-11-27T16:15:07.177 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07700693d0 0x7f0770158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.177 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07700693d0 0x7f0770158230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36868/0 (socket says 172.21.15.164:36868) 2022-11-27T16:15:07.177 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 -- 172.21.15.164:0/2086319202 learned_addr learned my addr 172.21.15.164:0/2086319202 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:07.177 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076f7fe700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07700ae0d0 0x7f07700b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f0774816700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f077015f2d0 0x7f07700b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 -- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f077015f2d0 msgr2=0x7f07700b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f077015f2d0 0x7f07700b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.178 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 -- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07700ae0d0 msgr2=0x7f07700b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07700ae0d0 0x7f07700b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 -- 172.21.15.164:0/2086319202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600094a0 con 0x7f07700693d0 2022-11-27T16:15:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076ffff700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07700693d0 0x7f0770158230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f076000ebe0 tx=0x7f076000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.179 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076cff9700 1 -- 172.21.15.164:0/2086319202 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000c850 con 0x7f07700693d0 2022-11-27T16:15:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076cff9700 1 -- 172.21.15.164:0/2086319202 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076000c9b0 con 0x7f07700693d0 2022-11-27T16:15:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07580094a0 con 0x7f07700693d0 2022-11-27T16:15:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f076cff9700 1 -- 172.21.15.164:0/2086319202 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000cd40 con 0x7f07700693d0 2022-11-27T16:15:07.180 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.164+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07700c2db0 con 0x7f07700693d0 2022-11-27T16:15:07.181 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.165+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0770073080 con 0x7f07700693d0 2022-11-27T16:15:07.181 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.166+0000 7f076cff9700 1 -- 172.21.15.164:0/2086319202 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f0760019030 con 0x7f07700693d0 2022-11-27T16:15:07.182 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.166+0000 7f076cff9700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f0754066590 0x7f0754068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.182 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.166+0000 7f076cff9700 1 -- 172.21.15.164:0/2086319202 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0760083420 con 0x7f07700693d0 2022-11-27T16:15:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.166+0000 7f076f7fe700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f0754066590 0x7f0754068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.183 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.166+0000 7f076f7fe700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f0754066590 0x7f0754068a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0758000c00 tx=0x7f0758008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.184 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.170+0000 7f076cff9700 1 -- 172.21.15.164:0/2086319202 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f0760050e80 con 0x7f07700693d0 2022-11-27T16:15:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.349+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f07700b1c10 con 0x7f07700693d0 2022-11-27T16:15:07.351 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.350+0000 7f076cff9700 1 -- 172.21.15.164:0/2086319202 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f076000cea0 con 0x7f07700693d0 2022-11-27T16:15:07.352 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 8 2022-11-27T16:15:07.353 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f0754066590 msgr2=0x7f0754068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f0754066590 0x7f0754068a40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0758000c00 tx=0x7f0758008030 comp rx=0 tx=0).stop 2022-11-27T16:15:07.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07700693d0 msgr2=0x7f0770158230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07700693d0 0x7f0770158230 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f076000ebe0 tx=0x7f076000eef0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 shutdown_connections 2022-11-27T16:15:07.354 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f0754066590 0x7f0754068a40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f077015f2d0 0x7f07700b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f07700693d0 0x7f0770158230 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 --2- 172.21.15.164:0/2086319202 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f07700ae0d0 0x7f07700b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.355 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 >> 172.21.15.164:0/2086319202 conn(0x7f077006ce40 msgr2=0x7f077015f730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:07.356 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 shutdown_connections 2022-11-27T16:15:07.356 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.352+0000 7f0776279700 1 -- 172.21.15.164:0/2086319202 wait complete. 2022-11-27T16:15:07.365 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T16:15:07.490 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.488+0000 7f81e3be0700 1 Processor -- start 2022-11-27T16:15:07.490 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.488+0000 7f81e3be0700 1 -- start start 2022-11-27T16:15:07.496 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e3be0700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f81dc06e840 0x7f81dc06eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.496 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e3be0700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc0cd480 con 0x7f81dc0b0710 2022-11-27T16:15:07.496 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e3be0700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc15c570 con 0x7f81dc0cd0b0 2022-11-27T16:15:07.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e3be0700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81dc0ae5b0 con 0x7f81dc06e840 2022-11-27T16:15:07.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e217d700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f81dc06e840 0x7f81dc06eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.497 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e117b700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f81dc0b0710 0x7f81dc15fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59682/0 (socket says 172.21.15.164:59682) 2022-11-27T16:15:07.498 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e117b700 1 -- 172.21.15.164:0/1224335583 learned_addr learned my addr 172.21.15.164:0/1224335583 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:15:07.498 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e217d700 1 -- 172.21.15.164:0/1224335583 >> v1:172.21.15.167:6789/0 conn(0x7f81dc0cd0b0 legacy=0x7f81dc15bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:07.499 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e217d700 1 -- 172.21.15.164:0/1224335583 >> v1:172.21.15.164:6789/0 conn(0x7f81dc0b0710 legacy=0x7f81dc15fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.499 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e217d700 1 -- 172.21.15.164:0/1224335583 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc15c850 con 0x7f81dc06e840 2022-11-27T16:15:07.500 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e097a700 1 -- 172.21.15.164:0/1224335583 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879462467 0 0) 0x7f81dc0cd480 con 0x7f81dc0b0710 2022-11-27T16:15:07.500 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.489+0000 7f81e217d700 1 --2- 172.21.15.164:0/1224335583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc06eca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009d90 tx=0x7f81d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa852e61f13e4a6e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.500 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e097a700 1 -- 172.21.15.164:0/1224335583 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d400e070 con 0x7f81dc06e840 2022-11-27T16:15:07.501 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e097a700 1 -- 172.21.15.164:0/1224335583 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d400bb90 con 0x7f81dc06e840 2022-11-27T16:15:07.501 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e097a700 1 -- 172.21.15.164:0/1224335583 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4003a20 con 0x7f81dc06e840 2022-11-27T16:15:07.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e3be0700 1 -- 172.21.15.164:0/1224335583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 msgr2=0x7f81dc06eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1224335583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc06eca0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009d90 tx=0x7f81d400b330 comp rx=0 tx=0).stop 2022-11-27T16:15:07.502 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e3be0700 1 -- 172.21.15.164:0/1224335583 shutdown_connections 2022-11-27T16:15:07.503 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1224335583 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc06eca0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.503 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e3be0700 1 -- 172.21.15.164:0/1224335583 >> 172.21.15.164:0/1224335583 conn(0x7f81dc06cdf0 msgr2=0x7f81dc06d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:07.503 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e3be0700 1 -- 172.21.15.164:0/1224335583 shutdown_connections 2022-11-27T16:15:07.504 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.490+0000 7f81e3be0700 1 -- 172.21.15.164:0/1224335583 wait complete. 2022-11-27T16:15:07.505 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.491+0000 7f81e3be0700 1 Processor -- start 2022-11-27T16:15:07.505 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.491+0000 7f81e3be0700 1 -- start start 2022-11-27T16:15:07.505 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e3be0700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc0af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.506 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e3be0700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0b0710 0x7f81dc147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.506 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e3be0700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc0cd0b0 0x7f81dc0c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.506 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e3be0700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0ad4b0 con 0x7f81dc06e840 2022-11-27T16:15:07.507 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e3be0700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0c4ea0 con 0x7f81dc0b0710 2022-11-27T16:15:07.507 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e3be0700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0c5180 con 0x7f81dc0cd0b0 2022-11-27T16:15:07.508 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e197c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.508 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e197c700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc0af420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36886/0 (socket says 172.21.15.164:36886) 2022-11-27T16:15:07.509 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e197c700 1 -- 172.21.15.164:0/1635494106 learned_addr learned my addr 172.21.15.164:0/1635494106 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:07.509 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e117b700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0b0710 0x7f81dc147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.510 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e197c700 1 -- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc0cd0b0 msgr2=0x7f81dc0c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.510 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.492+0000 7f81e197c700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc0cd0b0 0x7f81dc0c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.510 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81e197c700 1 -- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0b0710 msgr2=0x7f81dc147c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.511 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81e197c700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0b0710 0x7f81dc147c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.511 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81e197c700 1 -- 172.21.15.164:0/1635494106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4009a70 con 0x7f81dc06e840 2022-11-27T16:15:07.512 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81e117b700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0b0710 0x7f81dc147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:15:07.512 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81e197c700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc0af420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00b470 tx=0x7f81cc00b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81ca7fc700 1 -- 172.21.15.164:0/1635494106 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc011830 con 0x7f81dc06e840 2022-11-27T16:15:07.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81ca7fc700 1 -- 172.21.15.164:0/1635494106 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc011990 con 0x7f81dc06e840 2022-11-27T16:15:07.513 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc0c5460 con 0x7f81dc06e840 2022-11-27T16:15:07.514 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81ca7fc700 1 -- 172.21.15.164:0/1635494106 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc011d40 con 0x7f81dc06e840 2022-11-27T16:15:07.514 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.493+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc0c5a20 con 0x7f81dc06e840 2022-11-27T16:15:07.514 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.494+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc073080 con 0x7f81dc06e840 2022-11-27T16:15:07.515 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.494+0000 7f81ca7fc700 1 -- 172.21.15.164:0/1635494106 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f81cc021070 con 0x7f81dc06e840 2022-11-27T16:15:07.515 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.495+0000 7f81ca7fc700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f81bc066590 0x7f81bc068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.515 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.495+0000 7f81ca7fc700 1 -- 172.21.15.164:0/1635494106 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f81cc083380 con 0x7f81dc06e840 2022-11-27T16:15:07.516 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.495+0000 7f81e117b700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f81bc066590 0x7f81bc068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.519 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.495+0000 7f81e117b700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f81bc066590 0x7f81bc068a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81dc15dd60 tx=0x7f81d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.520 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.499+0000 7f81ca7fc700 1 -- 172.21.15.164:0/1635494106 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f81cc050de0 con 0x7f81dc06e840 2022-11-27T16:15:07.724 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.722+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81dc0b1250 con 0x7f81dc06e840 2022-11-27T16:15:07.725 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.723+0000 7f81ca7fc700 1 -- 172.21.15.164:0/1635494106 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7f81cc015de0 con 0x7f81dc06e840 2022-11-27T16:15:07.725 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:15:07.725 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6835/2160956746","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6833","nonce":2160956746},{"type":"v1","addr":"172.21.15.167:6835","nonce":2160956746}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4278,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6837/428518809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6835","nonce":428518809},{"type":"v1","addr":"172.21.15.164:6837","nonce":428518809}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.164:6836/201174558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6834","nonce":201174558},{"type":"v1","addr":"172.21.15.164:6836","nonce":201174558}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4312,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.167:6834/1239858278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6832","nonce":1239858278},{"type":"v1","addr":"172.21.15.167:6834","nonce":1239858278}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-11-27T16:15:07.727 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f81bc066590 msgr2=0x7f81bc068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.728 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f81bc066590 0x7f81bc068a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81dc15dd60 tx=0x7f81d8009040 comp rx=0 tx=0).stop 2022-11-27T16:15:07.728 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 msgr2=0x7f81dc0af420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.728 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc0af420 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00b470 tx=0x7f81cc00b830 comp rx=0 tx=0).stop 2022-11-27T16:15:07.728 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 shutdown_connections 2022-11-27T16:15:07.728 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f81bc066590 0x7f81bc068a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.729 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f81dc0cd0b0 0x7f81dc0c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.729 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f81dc06e840 0x7f81dc0af420 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.729 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 --2- 172.21.15.164:0/1635494106 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f81dc0b0710 0x7f81dc147c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.729 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 >> 172.21.15.164:0/1635494106 conn(0x7f81dc06cdf0 msgr2=0x7f81dc15b100 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:07.730 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.725+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 shutdown_connections 2022-11-27T16:15:07.730 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.726+0000 7f81e3be0700 1 -- 172.21.15.164:0/1635494106 wait complete. 2022-11-27T16:15:07.730 INFO:teuthology.orchestra.run.smithi164.stderr:dumped fsmap epoch 3 2022-11-27T16:15:07.740 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:15:07.861 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.859+0000 7fbcdf6aa700 1 Processor -- start 2022-11-27T16:15:07.861 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdf6aa700 1 -- start start 2022-11-27T16:15:07.864 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdf6aa700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7fbcd815bb00 0x7fbcd80ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.865 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdf6aa700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbcd80af2d0 0x7fbcd80cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.865 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdf6aa700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd806a4e0 0x7fbcd806a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.865 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdf6aa700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd806b0a0 con 0x7fbcd80af2d0 2022-11-27T16:15:07.865 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdf6aa700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd806aea0 con 0x7fbcd815bb00 2022-11-27T16:15:07.866 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdf6aa700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80b06a0 con 0x7fbcd806a4e0 2022-11-27T16:15:07.866 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.860+0000 7fbcdcc45700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbcd80af2d0 0x7fbcd80cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.866 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7fbcd80af2d0 0x7fbcd80cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36894/0 (socket says 172.21.15.164:36894) 2022-11-27T16:15:07.866 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 -- 172.21.15.164:0/774746666 learned_addr learned my addr 172.21.15.164:0/774746666 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:07.866 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcddc47700 1 --2- 172.21.15.164:0/774746666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd806a4e0 0x7fbcd806a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.867 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdd446700 1 --2- 172.21.15.164:0/774746666 >> v2:172.21.15.167:3300/0 conn(0x7fbcd815bb00 0x7fbcd80ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.867 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 -- 172.21.15.164:0/774746666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd806a4e0 msgr2=0x7fbcd806a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:07.867 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 --2- 172.21.15.164:0/774746666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd806a4e0 0x7fbcd806a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.867 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 -- 172.21.15.164:0/774746666 >> v2:172.21.15.167:3300/0 conn(0x7fbcd815bb00 msgr2=0x7fbcd80ae470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:07.868 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 --2- 172.21.15.164:0/774746666 >> v2:172.21.15.167:3300/0 conn(0x7fbcd815bb00 0x7fbcd80ae470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.868 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 -- 172.21.15.164:0/774746666 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80af7b0 con 0x7fbcd80af2d0 2022-11-27T16:15:07.868 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdd446700 1 --2- 172.21.15.164:0/774746666 >> v2:172.21.15.167:3300/0 conn(0x7fbcd815bb00 0x7fbcd80ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:15:07.868 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcddc47700 1 --2- 172.21.15.164:0/774746666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd806a4e0 0x7fbcd806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:15:07.869 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcdcc45700 1 --2- 172.21.15.164:0/774746666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd80af2d0 0x7fbcd80cda60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2a80 tx=0x7fbcc8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883afd0a108b9a2c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.869 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcc7fff700 1 -- 172.21.15.164:0/774746666 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8005e30 con 0x7fbcd80af2d0 2022-11-27T16:15:07.869 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcc7fff700 1 -- 172.21.15.164:0/774746666 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc8004030 con 0x7fbcd80af2d0 2022-11-27T16:15:07.869 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.861+0000 7fbcc7fff700 1 -- 172.21.15.164:0/774746666 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80043e0 con 0x7fbcd80af2d0 2022-11-27T16:15:07.869 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/774746666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd80af2d0 msgr2=0x7fbcd80cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.870 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/774746666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd80af2d0 0x7fbcd80cda60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2a80 tx=0x7fbcc8009bd0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.870 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/774746666 shutdown_connections 2022-11-27T16:15:07.870 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/774746666 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd806a4e0 0x7fbcd806a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.870 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/774746666 >> v2:172.21.15.167:3300/0 conn(0x7fbcd815bb00 0x7fbcd80ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.870 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/774746666 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd80af2d0 0x7fbcd80cda60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.871 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/774746666 >> 172.21.15.164:0/774746666 conn(0x7fbcd8099160 msgr2=0x7fbcd80cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:07.871 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/774746666 shutdown_connections 2022-11-27T16:15:07.871 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.862+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/774746666 wait complete. 2022-11-27T16:15:07.871 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.863+0000 7fbcdf6aa700 1 Processor -- start 2022-11-27T16:15:07.872 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.863+0000 7fbcdf6aa700 1 -- start start 2022-11-27T16:15:07.877 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdf6aa700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd806a4e0 0x7fbcd81f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.878 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdf6aa700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd80af2d0 0x7fbcd8159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.878 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdf6aa700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcd815bb00 0x7fbcd80ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.879 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdf6aa700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd81f1320 con 0x7fbcd806a4e0 2022-11-27T16:15:07.879 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdf6aa700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd815a270 con 0x7fbcd815bb00 2022-11-27T16:15:07.879 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdf6aa700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd815a550 con 0x7fbcd80af2d0 2022-11-27T16:15:07.880 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd806a4e0 0x7fbcd81f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.880 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd806a4e0 0x7fbcd81f0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36900/0 (socket says 172.21.15.164:36900) 2022-11-27T16:15:07.880 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 -- 172.21.15.164:0/3540826576 learned_addr learned my addr 172.21.15.164:0/3540826576 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:07.881 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdcc45700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd80af2d0 0x7fbcd8159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.881 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcddc47700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcd815bb00 0x7fbcd80ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.881 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 -- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd80af2d0 msgr2=0x7fbcd8159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.881 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd80af2d0 0x7fbcd8159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.882 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 -- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcd815bb00 msgr2=0x7fbcd80ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:07.882 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcd815bb00 0x7fbcd80ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:07.882 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 -- 172.21.15.164:0/3540826576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8009500 con 0x7fbcd806a4e0 2022-11-27T16:15:07.882 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcddc47700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcd815bb00 0x7fbcd80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:15:07.882 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdd446700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd806a4e0 0x7fbcd81f0de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400e810 tx=0x7fbcd400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.883 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcc5ffb700 1 -- 172.21.15.164:0/3540826576 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd4004230 con 0x7fbcd806a4e0 2022-11-27T16:15:07.883 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcc5ffb700 1 -- 172.21.15.164:0/3540826576 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd40043b0 con 0x7fbcd806a4e0 2022-11-27T16:15:07.883 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd815a830 con 0x7fbcd806a4e0 2022-11-27T16:15:07.883 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.864+0000 7fbcc5ffb700 1 -- 172.21.15.164:0/3540826576 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd40038e0 con 0x7fbcd806a4e0 2022-11-27T16:15:07.884 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.865+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd80b4060 con 0x7fbcd806a4e0 2022-11-27T16:15:07.884 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.866+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd80b4410 con 0x7fbcd806a4e0 2022-11-27T16:15:07.884 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.866+0000 7fbcc5ffb700 1 -- 172.21.15.164:0/3540826576 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fbcd4022070 con 0x7fbcd806a4e0 2022-11-27T16:15:07.884 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.866+0000 7fbcc5ffb700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fbcb8066590 0x7fbcb8068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:07.885 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.866+0000 7fbcc5ffb700 1 -- 172.21.15.164:0/3540826576 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fbcd4082e30 con 0x7fbcd806a4e0 2022-11-27T16:15:07.885 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.866+0000 7fbcdcc45700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fbcb8066590 0x7fbcb8068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:07.885 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.867+0000 7fbcdcc45700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fbcb8066590 0x7fbcb8068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2a80 tx=0x7fbcc8008070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:07.885 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:07.870+0000 7fbcc5ffb700 1 -- 172.21.15.164:0/3540826576 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fbcd4050890 con 0x7fbcd806a4e0 2022-11-27T16:15:08.052 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.051+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcd80b41f0 con 0x7fbcd806a4e0 2022-11-27T16:15:08.053 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.052+0000 7fbcc5ffb700 1 -- 172.21.15.164:0/3540826576 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13789 (secure 0 0 0) 0x7fbcd400f040 con 0x7fbcd806a4e0 2022-11-27T16:15:08.054 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:15:08.054 INFO:teuthology.orchestra.run.smithi164.stdout:{"epoch":19,"fsid":"94c9dd9b-1805-4f9e-833a-3b29514744e3","created":"2022-11-27T16:14:46.055229+0000","modified":"2022-11-27T16:15:07.333930+0000","last_up_change":"2022-11-27T16:14:57.299464+0000","last_in_change":"2022-11-27T16:14:48.616289+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:14:57.480549+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T16:15:04.534458+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"fc87cc2c-4aba-41f8-9b94-8843abd34341","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6800","nonce":101367},{"type":"v1","addr":"172.21.15.164:6801","nonce":101367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6802","nonce":101367},{"type":"v1","addr":"172.21.15.164:6803","nonce":101367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6806","nonce":101367},{"type":"v1","addr":"172.21.15.164:6807","nonce":101367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6804","nonce":101367},{"type":"v1","addr":"172.21.15.164:6805","nonce":101367}]},"public_addr":"172.21.15.164:6801/101367","cluster_addr":"172.21.15.164:6803/101367","heartbeat_back_addr":"172.21.15.164:6807/101367","heartbeat_front_addr":"172.21.15.164:6805/101367","state":["exists","up"]},{"osd":1,"uuid":"2288745b-11fa-433b-bf88-dbfd0693401d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6808","nonce":101368},{"type":"v1","addr":"172.21.15.164:6810","nonce":101368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6812","nonce":101368},{"type":"v1","addr":"172.21.15.164:6814","nonce":101368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6820","nonce":101368},{"type":"v1","addr":"172.21.15.164:6822","nonce":101368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6816","nonce":101368},{"type":"v1","addr":"172.21.15.164:6818","nonce":101368}]},"public_addr":"172.21.15.164:6810/101368","cluster_addr":"172.21.15.164:6814/101368","heartbeat_back_addr":"172.21.15.164:6822/101368","heartbeat_front_addr":"172.21.15.164:6818/101368","state":["exists","up"]},{"osd":2,"uuid":"82ed3e90-5d84-4fc3-ac13-a6a146bc60f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6824","nonce":101377},{"type":"v1","addr":"172.21.15.164:6825","nonce":101377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6826","nonce":101377},{"type":"v1","addr":"172.21.15.164:6827","nonce":101377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6830","nonce":101377},{"type":"v1","addr":"172.21.15.164:6831","nonce":101377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6828","nonce":101377},{"type":"v1","addr":"172.21.15.164:6829","nonce":101377}]},"public_addr":"172.21.15.164:6825/101377","cluster_addr":"172.21.15.164:6827/101377","heartbeat_back_addr":"172.21.15.164:6831/101377","heartbeat_front_addr":"172.21.15.164:6829/101377","state":["exists","up"]},{"osd":3,"uuid":"864ca05d-4b9e-4111-8130-dd230c4a663c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6809","nonce":101392},{"type":"v1","addr":"172.21.15.164:6811","nonce":101392}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6813","nonce":101392},{"type":"v1","addr":"172.21.15.164:6815","nonce":101392}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6821","nonce":101392},{"type":"v1","addr":"172.21.15.164:6823","nonce":101392}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.164:6817","nonce":101392},{"type":"v1","addr":"172.21.15.164:6819","nonce":101392}]},"public_addr":"172.21.15.164:6811/101392","cluster_addr":"172.21.15.164:6815/101392","heartbeat_back_addr":"172.21.15.164:6823/101392","heartbeat_front_addr":"172.21.15.164:6819/101392","state":["exists","up"]},{"osd":4,"uuid":"94560d98-a6fa-494f-8407-d432c650a7f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6808","nonce":101633},{"type":"v1","addr":"172.21.15.167:6809","nonce":101633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6810","nonce":101633},{"type":"v1","addr":"172.21.15.167:6811","nonce":101633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6814","nonce":101633},{"type":"v1","addr":"172.21.15.167:6815","nonce":101633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6812","nonce":101633},{"type":"v1","addr":"172.21.15.167:6813","nonce":101633}]},"public_addr":"172.21.15.167:6809/101633","cluster_addr":"172.21.15.167:6811/101633","heartbeat_back_addr":"172.21.15.167:6815/101633","heartbeat_front_addr":"172.21.15.167:6813/101633","state":["exists","up"]},{"osd":5,"uuid":"815803f1-a95d-4921-a083-8185392357c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6817","nonce":101647},{"type":"v1","addr":"172.21.15.167:6819","nonce":101647}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6820","nonce":101647},{"type":"v1","addr":"172.21.15.167:6822","nonce":101647}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6827","nonce":101647},{"type":"v1","addr":"172.21.15.167:6829","nonce":101647}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6824","nonce":101647},{"type":"v1","addr":"172.21.15.167:6825","nonce":101647}]},"public_addr":"172.21.15.167:6819/101647","cluster_addr":"172.21.15.167:6822/101647","heartbeat_back_addr":"172.21.15.167:6829/101647","heartbeat_front_addr":"172.21.15.167:6825/101647","state":["exists","up"]},{"osd":6,"uuid":"7e0d4655-210d-4601-8e1f-67a9fa7452ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6816","nonce":101652},{"type":"v1","addr":"172.21.15.167:6818","nonce":101652}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6821","nonce":101652},{"type":"v1","addr":"172.21.15.167:6823","nonce":101652}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6830","nonce":101652},{"type":"v1","addr":"172.21.15.167:6831","nonce":101652}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6826","nonce":101652},{"type":"v1","addr":"172.21.15.167:6828","nonce":101652}]},"public_addr":"172.21.15.167:6818/101652","cluster_addr":"172.21.15.167:6823/101652","heartbeat_back_addr":"172.21.15.167:6831/101652","heartbeat_front_addr":"172.21.15.167:6828/101652","state":["exists","up"]},{"osd":7,"uuid":"469ab427-7d6f-4f2a-9d32-275b76638d81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6800","nonce":101634},{"type":"v1","addr":"172.21.15.167:6801","nonce":101634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6802","nonce":101634},{"type":"v1","addr":"172.21.15.167:6803","nonce":101634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6806","nonce":101634},{"type":"v1","addr":"172.21.15.167:6807","nonce":101634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.167:6804","nonce":101634},{"type":"v1","addr":"172.21.15.167:6805","nonce":101634}]},"public_addr":"172.21.15.167:6801/101634","cluster_addr":"172.21.15.167:6803/101634","heartbeat_back_addr":"172.21.15.167:6807/101634","heartbeat_front_addr":"172.21.15.167:6805/101634","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.591828+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.746590+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.798994+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.821213+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.864723+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.826163+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.784349+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:14:53.812065+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:15:08.056 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fbcb8066590 msgr2=0x7fbcb8068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.056 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fbcb8066590 0x7fbcb8068a40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2a80 tx=0x7fbcc8008070 comp rx=0 tx=0).stop 2022-11-27T16:15:08.056 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd806a4e0 msgr2=0x7fbcd81f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.056 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd806a4e0 0x7fbcd81f0de0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400e810 tx=0x7fbcd400eb20 comp rx=0 tx=0).stop 2022-11-27T16:15:08.057 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 shutdown_connections 2022-11-27T16:15:08.057 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7fbcb8066590 0x7fbcb8068a40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.057 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7fbcd815bb00 0x7fbcd80ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.057 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7fbcd806a4e0 0x7fbcd81f0de0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.058 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 --2- 172.21.15.164:0/3540826576 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7fbcd80af2d0 0x7fbcd8159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.058 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 >> 172.21.15.164:0/3540826576 conn(0x7fbcd8099160 msgr2=0x7fbcd80adc90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:08.058 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 shutdown_connections 2022-11-27T16:15:08.058 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.054+0000 7fbcdf6aa700 1 -- 172.21.15.164:0/3540826576 wait complete. 2022-11-27T16:15:08.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:14:57.480549+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:15:04.534458+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T16:15:08.068 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:15:08.191 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.189+0000 7f480bba7700 1 Processor -- start 2022-11-27T16:15:08.192 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.190+0000 7f480bba7700 1 -- start start 2022-11-27T16:15:08.206 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f480bba7700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48040af110 0x7f48040b2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.207 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f480bba7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040ca420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.207 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f480bba7700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48040ce3b0 con 0x7f48040b1850 2022-11-27T16:15:08.208 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f480bba7700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48040cbf10 con 0x7f48040af110 2022-11-27T16:15:08.208 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f480bba7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48040cf9d0 con 0x7f48040cf460 2022-11-27T16:15:08.209 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809142700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.209 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809943700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48040af110 0x7f48040b2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809943700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f48040af110 0x7f48040b2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.164:38146/0 (socket says 172.21.15.164:38146) 2022-11-27T16:15:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809142700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040ca420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.164:51616/0 (socket says 172.21.15.164:51616) 2022-11-27T16:15:08.210 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809943700 1 -- 172.21.15.164:0/1492118146 learned_addr learned my addr 172.21.15.164:0/1492118146 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:08.211 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809142700 1 -- 172.21.15.164:0/1492118146 >> v2:172.21.15.167:3300/0 conn(0x7f48040af110 msgr2=0x7f48040b2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:08.211 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809142700 1 --2- 172.21.15.164:0/1492118146 >> v2:172.21.15.167:3300/0 conn(0x7f48040af110 0x7f48040b2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.212 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809142700 1 -- 172.21.15.164:0/1492118146 >> v1:172.21.15.167:6789/0 conn(0x7f48040b1850 legacy=0x7f48040cc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.212 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.191+0000 7f4809142700 1 -- 172.21.15.164:0/1492118146 --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48040ce600 con 0x7f48040cf460 2022-11-27T16:15:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f4809142700 1 --2- 172.21.15.164:0/1492118146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040ca420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4800009840 tx=0x7f4800009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac8604d4f33922ba server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.213 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f4808941700 1 -- 172.21.15.164:0/1492118146 <== mon.2 v2:172.21.15.167:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800011830 con 0x7f48040cf460 2022-11-27T16:15:08.214 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f4808941700 1 -- 172.21.15.164:0/1492118146 <== mon.2 v2:172.21.15.167:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4800011990 con 0x7f48040cf460 2022-11-27T16:15:08.214 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f4808941700 1 -- 172.21.15.164:0/1492118146 <== mon.2 v2:172.21.15.167:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800011d20 con 0x7f48040cf460 2022-11-27T16:15:08.214 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 -- 172.21.15.164:0/1492118146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 msgr2=0x7f48040ca420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.215 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 --2- 172.21.15.164:0/1492118146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040ca420 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4800009840 tx=0x7f4800009b50 comp rx=0 tx=0).stop 2022-11-27T16:15:08.215 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 -- 172.21.15.164:0/1492118146 shutdown_connections 2022-11-27T16:15:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 --2- 172.21.15.164:0/1492118146 >> v2:172.21.15.167:3300/0 conn(0x7f48040af110 0x7f48040b2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.216 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 --2- 172.21.15.164:0/1492118146 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040ca420 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 -- 172.21.15.164:0/1492118146 >> 172.21.15.164:0/1492118146 conn(0x7f480406df40 msgr2=0x7f480406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:08.217 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 -- 172.21.15.164:0/1492118146 shutdown_connections 2022-11-27T16:15:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.192+0000 7f480bba7700 1 -- 172.21.15.164:0/1492118146 wait complete. 2022-11-27T16:15:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 Processor -- start 2022-11-27T16:15:08.218 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 -- start start 2022-11-27T16:15:08.219 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48040af110 0x7f480414d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.219 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48040b1850 0x7f48040b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.220 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040b53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.220 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48041f1440 con 0x7f48040af110 2022-11-27T16:15:08.221 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48040c6c40 con 0x7f48040b1850 2022-11-27T16:15:08.221 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f480bba7700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480406a580 con 0x7f48040cf460 2022-11-27T16:15:08.222 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f4809943700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48040af110 0x7f480414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.222 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f4809943700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48040af110 0x7f480414d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36914/0 (socket says 172.21.15.164:36914) 2022-11-27T16:15:08.222 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.194+0000 7f4809943700 1 -- 172.21.15.164:0/1944186547 learned_addr learned my addr 172.21.15.164:0/1944186547 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:08.223 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f4809142700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48040b1850 0x7f48040b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.223 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f480a144700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040b53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f4809943700 1 -- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 msgr2=0x7f48040b53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.224 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f4809943700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040b53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.225 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f4809943700 1 -- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48040b1850 msgr2=0x7f48040b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.225 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f4809943700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48040b1850 0x7f48040b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f4809943700 1 -- 172.21.15.164:0/1944186547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48000094a0 con 0x7f48040af110 2022-11-27T16:15:08.226 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f4809943700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48040af110 0x7f480414d110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47f400e8b0 tx=0x7f47f400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.227 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f47f27fc700 1 -- 172.21.15.164:0/1944186547 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4009600 con 0x7f48040af110 2022-11-27T16:15:08.227 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f47f27fc700 1 -- 172.21.15.164:0/1944186547 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4013680 con 0x7f48040af110 2022-11-27T16:15:08.228 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f480406a860 con 0x7f48040af110 2022-11-27T16:15:08.228 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f47f27fc700 1 -- 172.21.15.164:0/1944186547 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4013a30 con 0x7f48040af110 2022-11-27T16:15:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.195+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480406ae20 con 0x7f48040af110 2022-11-27T16:15:08.229 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.196+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f480406afe0 con 0x7f48040af110 2022-11-27T16:15:08.230 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.196+0000 7f47f27fc700 1 -- 172.21.15.164:0/1944186547 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f47f4019030 con 0x7f48040af110 2022-11-27T16:15:08.230 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.197+0000 7f47f27fc700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f47e4066590 0x7f47e4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.230 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.197+0000 7f4809142700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f47e4066590 0x7f47e4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.231 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.197+0000 7f47f27fc700 1 -- 172.21.15.164:0/1944186547 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f47f40831a0 con 0x7f48040af110 2022-11-27T16:15:08.231 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.197+0000 7f4809142700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f47e4066590 0x7f47e4068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4800009fd0 tx=0x7f480001c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.232 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.201+0000 7f47f27fc700 1 -- 172.21.15.164:0/1944186547 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f47f4050b80 con 0x7f48040af110 2022-11-27T16:15:08.384 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.382+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f480406b190 con 0x7f48040af110 2022-11-27T16:15:08.384 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.383+0000 7f47f27fc700 1 -- 172.21.15.164:0/1944186547 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47f40541a0 con 0x7f48040af110 2022-11-27T16:15:08.385 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 1 2022-11-27T16:15:08.387 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f47e4066590 msgr2=0x7f47e4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.387 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f47e4066590 0x7f47e4068a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4800009fd0 tx=0x7f480001c040 comp rx=0 tx=0).stop 2022-11-27T16:15:08.387 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48040af110 msgr2=0x7f480414d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.387 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48040af110 0x7f480414d110 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47f400e8b0 tx=0x7f47f400ebc0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.388 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 shutdown_connections 2022-11-27T16:15:08.388 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f47e4066590 0x7f47e4068a40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.388 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f48040cf460 0x7f48040b53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.389 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f48040af110 0x7f480414d110 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.389 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 --2- 172.21.15.164:0/1944186547 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f48040b1850 0x7f48040b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.389 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 >> 172.21.15.164:0/1944186547 conn(0x7f480406df40 msgr2=0x7f48040b30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:08.389 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 shutdown_connections 2022-11-27T16:15:08.390 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.385+0000 7f480bba7700 1 -- 172.21.15.164:0/1944186547 wait complete. 2022-11-27T16:15:08.398 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:15:08.522 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.521+0000 7f9da533a700 1 Processor -- start 2022-11-27T16:15:08.523 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.521+0000 7f9da533a700 1 -- start start 2022-11-27T16:15:08.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9da533a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da015b700 0x7f9da006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9da533a700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9da006b120 0x7f9da00cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9da533a700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da00cc6a0 con 0x7f9da00d1a20 2022-11-27T16:15:08.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9da533a700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9da00ccc60 con 0x7f9da006b120 2022-11-27T16:15:08.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9da533a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da006b5a0 con 0x7f9da015b700 2022-11-27T16:15:08.527 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9da006b120 0x7f9da00cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.527 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f9da006b120 0x7f9da00cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36930/0 (socket says 172.21.15.164:36930) 2022-11-27T16:15:08.527 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 -- 172.21.15.164:0/2839876190 learned_addr learned my addr 172.21.15.164:0/2839876190 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:08.527 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9effd700 1 --2- 172.21.15.164:0/2839876190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da015b700 0x7f9da006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 -- 172.21.15.164:0/2839876190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da015b700 msgr2=0x7f9da006abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:08.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 --2- 172.21.15.164:0/2839876190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da015b700 0x7f9da006abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 -- 172.21.15.164:0/2839876190 >> v1:172.21.15.167:6789/0 conn(0x7f9da00d1a20 legacy=0x7f9da00b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.528 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 -- 172.21.15.164:0/2839876190 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da00cc980 con 0x7f9da006b120 2022-11-27T16:15:08.529 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9effd700 1 --2- 172.21.15.164:0/2839876190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da015b700 0x7f9da006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:15:08.529 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9f7fe700 1 --2- 172.21.15.164:0/2839876190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da00cf100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009e50 tx=0x7f9d9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164df7685827ed6 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.529 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9dffb700 1 -- 172.21.15.164:0/2839876190 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800e070 con 0x7f9da006b120 2022-11-27T16:15:08.529 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9dffb700 1 -- 172.21.15.164:0/2839876190 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9800b9c0 con 0x7f9da006b120 2022-11-27T16:15:08.529 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.522+0000 7f9d9dffb700 1 -- 172.21.15.164:0/2839876190 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9800bd70 con 0x7f9da006b120 2022-11-27T16:15:08.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 -- 172.21.15.164:0/2839876190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 msgr2=0x7f9da00cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 --2- 172.21.15.164:0/2839876190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da00cf100 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d98009e50 tx=0x7f9d9800b330 comp rx=0 tx=0).stop 2022-11-27T16:15:08.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 -- 172.21.15.164:0/2839876190 shutdown_connections 2022-11-27T16:15:08.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 --2- 172.21.15.164:0/2839876190 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da00cf100 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 --2- 172.21.15.164:0/2839876190 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da015b700 0x7f9da006abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 -- 172.21.15.164:0/2839876190 >> 172.21.15.164:0/2839876190 conn(0x7f9da0070b00 msgr2=0x7f9da0070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:08.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 -- 172.21.15.164:0/2839876190 shutdown_connections 2022-11-27T16:15:08.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.523+0000 7f9da533a700 1 -- 172.21.15.164:0/2839876190 wait complete. 2022-11-27T16:15:08.532 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.524+0000 7f9da533a700 1 Processor -- start 2022-11-27T16:15:08.532 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.524+0000 7f9da533a700 1 -- start start 2022-11-27T16:15:08.535 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9da533a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da015e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9da533a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da00d1a20 0x7f9da00caac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9da533a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da015b700 0x7f9da00cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9da533a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da01f0df0 con 0x7f9da006b120 2022-11-27T16:15:08.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9da533a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da015caa0 con 0x7f9da015b700 2022-11-27T16:15:08.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9da533a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da015cd80 con 0x7f9da00d1a20 2022-11-27T16:15:08.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da015e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9effd700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da015e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36932/0 (socket says 172.21.15.164:36932) 2022-11-27T16:15:08.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9effd700 1 -- 172.21.15.164:0/2683744991 learned_addr learned my addr 172.21.15.164:0/2683744991 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:08.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9e7fc700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da00d1a20 0x7f9da00caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.538 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9f7fe700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da015b700 0x7f9da00cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.538 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9effd700 1 -- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da00d1a20 msgr2=0x7f9da00caac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.538 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9effd700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da00d1a20 0x7f9da00caac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.538 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9effd700 1 -- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da015b700 msgr2=0x7f9da00cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.538 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.525+0000 7f9d9effd700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da015b700 0x7f9da00cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.539 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.526+0000 7f9d9effd700 1 -- 172.21.15.164:0/2683744991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d98009b30 con 0x7f9da006b120 2022-11-27T16:15:08.539 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.526+0000 7f9d9effd700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da015e2a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000e810 tx=0x7f9d9000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.539 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.526+0000 7f9d877fe700 1 -- 172.21.15.164:0/2683744991 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9000c020 con 0x7f9da006b120 2022-11-27T16:15:08.539 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.526+0000 7f9d877fe700 1 -- 172.21.15.164:0/2683744991 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d90004250 con 0x7f9da006b120 2022-11-27T16:15:08.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.526+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da015d060 con 0x7f9da006b120 2022-11-27T16:15:08.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.526+0000 7f9d877fe700 1 -- 172.21.15.164:0/2683744991 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d900036a0 con 0x7f9da006b120 2022-11-27T16:15:08.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.526+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da015e7e0 con 0x7f9da006b120 2022-11-27T16:15:08.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.527+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da0073080 con 0x7f9da006b120 2022-11-27T16:15:08.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.527+0000 7f9d877fe700 1 -- 172.21.15.164:0/2683744991 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f9d90022070 con 0x7f9da006b120 2022-11-27T16:15:08.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.527+0000 7f9d877fe700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f9d80066590 0x7f9d80068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.528+0000 7f9d877fe700 1 -- 172.21.15.164:0/2683744991 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9d90083080 con 0x7f9da006b120 2022-11-27T16:15:08.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.528+0000 7f9d9e7fc700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f9d80066590 0x7f9d80068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.528+0000 7f9d9e7fc700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f9d80066590 0x7f9d80068a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9da014daf0 tx=0x7f9d94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.532+0000 7f9d877fe700 1 -- 172.21.15.164:0/2683744991 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f9d90050ae0 con 0x7f9da006b120 2022-11-27T16:15:08.710 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.708+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9da00d06c0 con 0x7f9da006b120 2022-11-27T16:15:08.710 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.709+0000 7f9d877fe700 1 -- 172.21.15.164:0/2683744991 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d90054100 con 0x7f9da006b120 2022-11-27T16:15:08.710 INFO:teuthology.orchestra.run.smithi164.stdout:pg_num: 8 2022-11-27T16:15:08.712 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f9d80066590 msgr2=0x7f9d80068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.713 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f9d80066590 0x7f9d80068a40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9da014daf0 tx=0x7f9d94009040 comp rx=0 tx=0).stop 2022-11-27T16:15:08.713 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 msgr2=0x7f9da015e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.714 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da015e2a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9d9000e810 tx=0x7f9d9000eb20 comp rx=0 tx=0).stop 2022-11-27T16:15:08.715 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 shutdown_connections 2022-11-27T16:15:08.715 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f9d80066590 0x7f9d80068a40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.715 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f9da015b700 0x7f9da00cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f9da006b120 0x7f9da015e2a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 --2- 172.21.15.164:0/2683744991 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f9da00d1a20 0x7f9da00caac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.710+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 >> 172.21.15.164:0/2683744991 conn(0x7f9da0070b00 msgr2=0x7f9da015a600 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:08.716 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.711+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 shutdown_connections 2022-11-27T16:15:08.717 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.711+0000 7f9da533a700 1 -- 172.21.15.164:0/2683744991 wait complete. 2022-11-27T16:15:08.724 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T16:15:08.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.842+0000 7f862235a700 1 Processor -- start 2022-11-27T16:15:08.844 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.842+0000 7f862235a700 1 -- start start 2022-11-27T16:15:08.850 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f862235a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c15b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.850 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f862235a700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f861c06d270 0x7f861c15f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.850 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f862235a700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861c15fba0 con 0x7f861c070d20 2022-11-27T16:15:08.850 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f862235a700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861c0ccd30 con 0x7f861c06d270 2022-11-27T16:15:08.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f862235a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c15fe80 con 0x7f861c15b4a0 2022-11-27T16:15:08.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f86208f7700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f861c070d20 0x7f861c0c8a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59744/0 (socket says 172.21.15.164:59744) 2022-11-27T16:15:08.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f861b7fe700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f861c06d270 0x7f861c15f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.851 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.843+0000 7f86208f7700 1 -- 172.21.15.164:0/1138931259 learned_addr learned my addr 172.21.15.164:0/1138931259 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:15:08.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861bfff700 1 --2- 172.21.15.164:0/1138931259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c15b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861b7fe700 1 -- 172.21.15.164:0/1138931259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 msgr2=0x7f861c15b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:08.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861b7fe700 1 --2- 172.21.15.164:0/1138931259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c15b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.852 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861b7fe700 1 -- 172.21.15.164:0/1138931259 >> v1:172.21.15.164:6789/0 conn(0x7f861c070d20 legacy=0x7f861c0c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861b7fe700 1 -- 172.21.15.164:0/1138931259 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c160160 con 0x7f861c06d270 2022-11-27T16:15:08.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861affd700 1 -- 172.21.15.164:0/1138931259 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142231357 0 0) 0x7f861c15fba0 con 0x7f861c070d20 2022-11-27T16:15:08.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861bfff700 1 --2- 172.21.15.164:0/1138931259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:15:08.853 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861b7fe700 1 --2- 172.21.15.164:0/1138931259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c15f370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f860c009840 tx=0x7f860c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96221ef6dbba902 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861affd700 1 -- 172.21.15.164:0/1138931259 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c011830 con 0x7f861c06d270 2022-11-27T16:15:08.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861affd700 1 -- 172.21.15.164:0/1138931259 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860c011990 con 0x7f861c06d270 2022-11-27T16:15:08.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f861affd700 1 -- 172.21.15.164:0/1138931259 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860c011d20 con 0x7f861c06d270 2022-11-27T16:15:08.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f862235a700 1 -- 172.21.15.164:0/1138931259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 msgr2=0x7f861c15f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.854 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f862235a700 1 --2- 172.21.15.164:0/1138931259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c15f370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f860c009840 tx=0x7f860c009b50 comp rx=0 tx=0).stop 2022-11-27T16:15:08.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f862235a700 1 -- 172.21.15.164:0/1138931259 shutdown_connections 2022-11-27T16:15:08.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f862235a700 1 --2- 172.21.15.164:0/1138931259 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c15b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.855 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f862235a700 1 --2- 172.21.15.164:0/1138931259 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c15f370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.844+0000 7f862235a700 1 -- 172.21.15.164:0/1138931259 >> 172.21.15.164:0/1138931259 conn(0x7f861c0ce410 msgr2=0x7f861c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:08.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.845+0000 7f862235a700 1 -- 172.21.15.164:0/1138931259 shutdown_connections 2022-11-27T16:15:08.856 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.845+0000 7f862235a700 1 -- 172.21.15.164:0/1138931259 wait complete. 2022-11-27T16:15:08.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.846+0000 7f862235a700 1 Processor -- start 2022-11-27T16:15:08.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.846+0000 7f862235a700 1 -- start start 2022-11-27T16:15:08.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f861c070d20 0x7f861c154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.857 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.858 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0b4060 con 0x7f861c06d270 2022-11-27T16:15:08.858 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0ae6c0 con 0x7f861c070d20 2022-11-27T16:15:08.858 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0aeb50 con 0x7f861c15b4a0 2022-11-27T16:15:08.858 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c151740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36946/0 (socket says 172.21.15.164:36946) 2022-11-27T16:15:08.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 -- 172.21.15.164:0/4037800270 learned_addr learned my addr 172.21.15.164:0/4037800270 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:08.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 -- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 msgr2=0x7f861c0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f86208f7700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.859 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861b7fe700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f861c070d20 0x7f861c154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.860 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c0b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.860 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 -- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f861c070d20 msgr2=0x7f861c154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:08.860 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f861c070d20 0x7f861c154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:08.860 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 -- 172.21.15.164:0/4037800270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0094a0 con 0x7f861c06d270 2022-11-27T16:15:08.861 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f861bfff700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c151740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f861000c7f0 tx=0x7f861000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.861 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f8618ff9700 1 -- 172.21.15.164:0/4037800270 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610011800 con 0x7f861c06d270 2022-11-27T16:15:08.861 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f8618ff9700 1 -- 172.21.15.164:0/4037800270 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8610011960 con 0x7f861c06d270 2022-11-27T16:15:08.861 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c0addd0 con 0x7f861c06d270 2022-11-27T16:15:08.862 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f8618ff9700 1 -- 172.21.15.164:0/4037800270 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8610011d10 con 0x7f861c06d270 2022-11-27T16:15:08.862 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.847+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861c0ae350 con 0x7f861c06d270 2022-11-27T16:15:08.862 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.848+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861c0ae9a0 con 0x7f861c06d270 2022-11-27T16:15:08.862 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.849+0000 7f8618ff9700 1 -- 172.21.15.164:0/4037800270 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f8610021070 con 0x7f861c06d270 2022-11-27T16:15:08.864 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.849+0000 7f8618ff9700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f85fc066590 0x7f85fc068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:08.864 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.849+0000 7f8618ff9700 1 -- 172.21.15.164:0/4037800270 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f86100832b0 con 0x7f861c06d270 2022-11-27T16:15:08.864 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.849+0000 7f861b7fe700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f85fc066590 0x7f85fc068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:08.865 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.849+0000 7f861b7fe700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f85fc066590 0x7f85fc068a40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f861c0b3650 tx=0x7f860c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:08.865 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:08.853+0000 7f8618ff9700 1 -- 172.21.15.164:0/4037800270 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f8610050d10 con 0x7f861c06d270 2022-11-27T16:15:09.036 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.034+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f861c073080 con 0x7f861c06d270 2022-11-27T16:15:09.036 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.035+0000 7f8618ff9700 1 -- 172.21.15.164:0/4037800270 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f8610054330 con 0x7f861c06d270 2022-11-27T16:15:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: 2022-11-27T16:15:09.037 INFO:teuthology.orchestra.run.smithi164.stdout:[ 2022-11-27T16:15:09.037 INFO:teuthology.orchestra.run.smithi164.stdout: { 2022-11-27T16:15:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 1, 2022-11-27T16:15:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": ".mgr" 2022-11-27T16:15:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: }, 2022-11-27T16:15:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: { 2022-11-27T16:15:09.038 INFO:teuthology.orchestra.run.smithi164.stdout: "poolnum": 2, 2022-11-27T16:15:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: "poolname": "rbd" 2022-11-27T16:15:09.039 INFO:teuthology.orchestra.run.smithi164.stdout: } 2022-11-27T16:15:09.039 INFO:teuthology.orchestra.run.smithi164.stdout:] 2022-11-27T16:15:09.040 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f85fc066590 msgr2=0x7f85fc068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:09.041 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f85fc066590 0x7f85fc068a40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f861c0b3650 tx=0x7f860c008030 comp rx=0 tx=0).stop 2022-11-27T16:15:09.041 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 msgr2=0x7f861c151740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:09.041 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c151740 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f861000c7f0 tx=0x7f861000cbb0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.041 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 shutdown_connections 2022-11-27T16:15:09.042 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f85fc066590 0x7f85fc068a40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.042 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f861c15b4a0 0x7f861c0b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.042 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f861c06d270 0x7f861c151740 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.042 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 --2- 172.21.15.164:0/4037800270 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f861c070d20 0x7f861c154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.042 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 >> 172.21.15.164:0/4037800270 conn(0x7f861c0ce410 msgr2=0x7f861c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:09.043 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 shutdown_connections 2022-11-27T16:15:09.043 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.037+0000 7f862235a700 1 -- 172.21.15.164:0/4037800270 wait complete. 2022-11-27T16:15:09.050 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T16:15:09.051 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T16:15:09.182 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.173+0000 7f290c5a3700 1 Processor -- start 2022-11-27T16:15:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.173+0000 7f290c5a3700 1 -- start start 2022-11-27T16:15:09.183 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290c5a3700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f290415b820 0x7f29040693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:09.192 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290c5a3700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2904069910 0x7f29040cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:09.193 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290c5a3700 1 -- --> v1:172.21.15.164:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29040b1270 con 0x7f29040cce30 2022-11-27T16:15:09.193 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290c5a3700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2904069d80 con 0x7f290415b820 2022-11-27T16:15:09.193 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290c5a3700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29040cd5b0 con 0x7f2904069910 2022-11-27T16:15:09.194 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290a33f700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f290415b820 0x7f29040693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:09.194 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290ab40700 1 --1- >> v1:172.21.15.164:6789/0 conn(0x7f29040cce30 0x7f290415f410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.164:6789/0 says I am v1:172.21.15.164:59758/0 (socket says 172.21.15.164:59758) 2022-11-27T16:15:09.195 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290ab40700 1 -- 172.21.15.164:0/3466529576 learned_addr learned my addr 172.21.15.164:0/3466529576 (peer_addr_for_me v1:172.21.15.164:0/0) 2022-11-27T16:15:09.195 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290a33f700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f290415b820 0x7f29040693d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36954/0 (socket says 172.21.15.164:36954) 2022-11-27T16:15:09.195 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f2909b3e700 1 --2- 172.21.15.164:0/3466529576 >> v2:172.21.15.167:3300/0 conn(0x7f2904069910 0x7f29040cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:09.196 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290a33f700 1 -- 172.21.15.164:0/3466529576 >> v2:172.21.15.167:3300/0 conn(0x7f2904069910 msgr2=0x7f29040cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:09.196 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290a33f700 1 --2- 172.21.15.164:0/3466529576 >> v2:172.21.15.167:3300/0 conn(0x7f2904069910 0x7f29040cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.197 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290a33f700 1 -- 172.21.15.164:0/3466529576 >> v1:172.21.15.164:6789/0 conn(0x7f29040cce30 legacy=0x7f290415f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:09.197 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290a33f700 1 -- 172.21.15.164:0/3466529576 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040b1700 con 0x7f290415b820 2022-11-27T16:15:09.197 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290933d700 1 -- 172.21.15.164:0/3466529576 <== mon.0 v1:172.21.15.164:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528791741 0 0) 0x7f29040b1270 con 0x7f29040cce30 2022-11-27T16:15:09.198 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.174+0000 7f290a33f700 1 --2- 172.21.15.164:0/3466529576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f290415b820 0x7f29040693d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f290415f370 tx=0x7f28f4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8477b57a0c7a36b0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:09.198 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290933d700 1 -- 172.21.15.164:0/3466529576 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4011830 con 0x7f290415b820 2022-11-27T16:15:09.198 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290933d700 1 -- 172.21.15.164:0/3466529576 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4011990 con 0x7f290415b820 2022-11-27T16:15:09.198 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290933d700 1 -- 172.21.15.164:0/3466529576 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4011d20 con 0x7f290415b820 2022-11-27T16:15:09.199 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 -- 172.21.15.164:0/3466529576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f290415b820 msgr2=0x7f29040693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:09.199 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 --2- 172.21.15.164:0/3466529576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f290415b820 0x7f29040693d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f290415f370 tx=0x7f28f4009b20 comp rx=0 tx=0).stop 2022-11-27T16:15:09.199 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 -- 172.21.15.164:0/3466529576 shutdown_connections 2022-11-27T16:15:09.199 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 --2- 172.21.15.164:0/3466529576 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f290415b820 0x7f29040693d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.200 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 --2- 172.21.15.164:0/3466529576 >> v2:172.21.15.167:3300/0 conn(0x7f2904069910 0x7f29040cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.200 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 -- 172.21.15.164:0/3466529576 >> 172.21.15.164:0/3466529576 conn(0x7f290406ce60 msgr2=0x7f290406d260 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:09.200 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 -- 172.21.15.164:0/3466529576 shutdown_connections 2022-11-27T16:15:09.200 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.175+0000 7f290c5a3700 1 -- 172.21.15.164:0/3466529576 wait complete. 2022-11-27T16:15:09.201 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.176+0000 7f290c5a3700 1 Processor -- start 2022-11-27T16:15:09.201 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.176+0000 7f290c5a3700 1 -- start start 2022-11-27T16:15:09.201 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290c5a3700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2904069910 0x7f2904159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:09.201 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290c5a3700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29040cce30 0x7f29040ab2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:09.201 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290c5a3700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290415b820 0x7f29040ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:09.202 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290c5a3700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29040c6f70 con 0x7f2904069910 2022-11-27T16:15:09.202 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290c5a3700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29040c7250 con 0x7f290415b820 2022-11-27T16:15:09.202 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290c5a3700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040c7530 con 0x7f29040cce30 2022-11-27T16:15:09.203 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2904069910 0x7f2904159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:09.203 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2904069910 0x7f2904159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36960/0 (socket says 172.21.15.164:36960) 2022-11-27T16:15:09.203 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 -- 172.21.15.164:0/4091548974 learned_addr learned my addr 172.21.15.164:0/4091548974 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:09.204 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290ab40700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290415b820 0x7f29040ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:09.204 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f2909b3e700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29040cce30 0x7f29040ab2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:09.204 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 -- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29040cce30 msgr2=0x7f29040ab2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:09.204 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29040cce30 0x7f29040ab2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.205 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 -- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290415b820 msgr2=0x7f29040ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:09.205 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290415b820 0x7f29040ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:09.205 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f290a33f700 1 -- 172.21.15.164:0/4091548974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40094a0 con 0x7f2904069910 2022-11-27T16:15:09.205 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.177+0000 7f2909b3e700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29040cce30 0x7f29040ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:15:09.205 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.178+0000 7f290a33f700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2904069910 0x7f2904159820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009b60 tx=0x7f28f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:09.206 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.178+0000 7f28f2ffd700 1 -- 172.21.15.164:0/4091548974 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f40038a0 con 0x7f2904069910 2022-11-27T16:15:09.206 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.178+0000 7f28f2ffd700 1 -- 172.21.15.164:0/4091548974 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f401e070 con 0x7f2904069910 2022-11-27T16:15:09.206 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.178+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29040c7810 con 0x7f2904069910 2022-11-27T16:15:09.206 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.178+0000 7f28f2ffd700 1 -- 172.21.15.164:0/4091548974 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4003c00 con 0x7f2904069910 2022-11-27T16:15:09.207 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.178+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29040ab800 con 0x7f2904069910 2022-11-27T16:15:09.207 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.179+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29040c7d30 con 0x7f2904069910 2022-11-27T16:15:09.207 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.179+0000 7f28f2ffd700 1 -- 172.21.15.164:0/4091548974 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f28f400f050 con 0x7f2904069910 2022-11-27T16:15:09.207 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.179+0000 7f28f2ffd700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f28e4066590 0x7f28e4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:09.207 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.180+0000 7f2909b3e700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f28e4066590 0x7f28e4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:09.208 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.180+0000 7f28f2ffd700 1 -- 172.21.15.164:0/4091548974 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f28f4084510 con 0x7f2904069910 2022-11-27T16:15:09.208 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.180+0000 7f2909b3e700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f28e4066590 0x7f28e4068a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29040ae3d0 tx=0x7f2900008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:09.209 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.184+0000 7f28f2ffd700 1 -- 172.21.15.164:0/4091548974 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f29040c7d30 con 0x7f2904069910 2022-11-27T16:15:09.365 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:09.363+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f29040c7d30 con 0x7f2904069910 2022-11-27T16:15:09.531 DEBUG:teuthology.orchestra.run.smithi164:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:15:09.534 DEBUG:teuthology.orchestra.run.smithi167:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:15:10.361 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.360+0000 7f28f2ffd700 1 -- 172.21.15.164:0/4091548974 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f29040c7d30 con 0x7f2904069910 2022-11-27T16:15:10.365 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.364+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f28e4066590 msgr2=0x7f28e4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:10.366 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.364+0000 7f290c5a3700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f28e4066590 0x7f28e4068a40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f29040ae3d0 tx=0x7f2900008040 comp rx=0 tx=0).stop 2022-11-27T16:15:10.366 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.364+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2904069910 msgr2=0x7f2904159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:10.366 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.364+0000 7f290c5a3700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2904069910 0x7f2904159820 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009b60 tx=0x7f28f4004750 comp rx=0 tx=0).stop 2022-11-27T16:15:10.367 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.365+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 shutdown_connections 2022-11-27T16:15:10.367 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.365+0000 7f290c5a3700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f28e4066590 0x7f28e4068a40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.365+0000 7f290c5a3700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f290415b820 0x7f29040ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.365+0000 7f290c5a3700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f2904069910 0x7f2904159820 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.365+0000 7f290c5a3700 1 --2- 172.21.15.164:0/4091548974 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f29040cce30 0x7f29040ab2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.368 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.365+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 >> 172.21.15.164:0/4091548974 conn(0x7f290406ce60 msgr2=0x7f29040b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:10.369 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.366+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 shutdown_connections 2022-11-27T16:15:10.369 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.366+0000 7f290c5a3700 1 -- 172.21.15.164:0/4091548974 wait complete. 2022-11-27T16:15:10.369 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_metadata' created 2022-11-27T16:15:10.383 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2022-11-27T16:15:10.516 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.515+0000 7f8a7e522700 1 Processor -- start 2022-11-27T16:15:10.517 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.515+0000 7f8a7e522700 1 -- start start 2022-11-27T16:15:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7e522700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f8a78159090 0x7f8a780ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7e522700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a7806c570 0x7f8a7806c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:10.520 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7e522700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a7806cef0 con 0x7f8a78159460 2022-11-27T16:15:10.521 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7e522700 1 -- --> v2:172.21.15.164:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a7815c4f0 con 0x7f8a7806c570 2022-11-27T16:15:10.521 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7e522700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a780ccb60 con 0x7f8a78159090 2022-11-27T16:15:10.521 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a7806c570 0x7f8a7806c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:10.521 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 --2- >> v2:172.21.15.164:3300/0 conn(0x7f8a7806c570 0x7f8a7806c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36972/0 (socket says 172.21.15.164:36972) 2022-11-27T16:15:10.522 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 -- 172.21.15.164:0/1662135086 learned_addr learned my addr 172.21.15.164:0/1662135086 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:10.522 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a77fff700 1 --2- 172.21.15.164:0/1662135086 >> v2:172.21.15.167:3300/0 conn(0x7f8a78159090 0x7f8a780ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:10.522 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 -- 172.21.15.164:0/1662135086 >> v2:172.21.15.167:3300/0 conn(0x7f8a78159090 msgr2=0x7f8a780ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:10.522 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 --2- 172.21.15.164:0/1662135086 >> v2:172.21.15.167:3300/0 conn(0x7f8a78159090 0x7f8a780ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.523 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 -- 172.21.15.164:0/1662135086 >> v1:172.21.15.167:6789/0 conn(0x7f8a78159460 legacy=0x7f8a7815bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:10.523 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 -- 172.21.15.164:0/1662135086 --> v2:172.21.15.164:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78070920 con 0x7f8a7806c570 2022-11-27T16:15:10.523 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a7cabf700 1 --2- 172.21.15.164:0/1662135086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a7806c570 0x7f8a7806c9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a7806d560 tx=0x7f8a7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f2cde1258c0643 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:10.523 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a76ffd700 1 -- 172.21.15.164:0/1662135086 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000e070 con 0x7f8a7806c570 2022-11-27T16:15:10.524 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a76ffd700 1 -- 172.21.15.164:0/1662135086 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7000b9c0 con 0x7f8a7806c570 2022-11-27T16:15:10.524 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.516+0000 7f8a76ffd700 1 -- 172.21.15.164:0/1662135086 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7000bd70 con 0x7f8a7806c570 2022-11-27T16:15:10.524 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 -- 172.21.15.164:0/1662135086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a7806c570 msgr2=0x7f8a7806c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:10.524 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 --2- 172.21.15.164:0/1662135086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a7806c570 0x7f8a7806c9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a7806d560 tx=0x7f8a7000b330 comp rx=0 tx=0).stop 2022-11-27T16:15:10.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 -- 172.21.15.164:0/1662135086 shutdown_connections 2022-11-27T16:15:10.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 --2- 172.21.15.164:0/1662135086 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a7806c570 0x7f8a7806c9b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 --2- 172.21.15.164:0/1662135086 >> v2:172.21.15.167:3300/0 conn(0x7f8a78159090 0x7f8a780ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 -- 172.21.15.164:0/1662135086 >> 172.21.15.164:0/1662135086 conn(0x7f8a780cbb40 msgr2=0x7f8a780ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:10.525 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 -- 172.21.15.164:0/1662135086 shutdown_connections 2022-11-27T16:15:10.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.517+0000 7f8a7e522700 1 -- 172.21.15.164:0/1662135086 wait complete. 2022-11-27T16:15:10.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.518+0000 7f8a7e522700 1 Processor -- start 2022-11-27T16:15:10.526 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.519+0000 7f8a7e522700 1 -- start start 2022-11-27T16:15:10.529 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7806c570 0x7f8a780c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:10.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 --2- >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78159090 0x7f8a781f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:10.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a78159460 0x7f8a78149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:10.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 -- --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7814ad10 con 0x7f8a78159460 2022-11-27T16:15:10.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 -- --> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780b2d60 con 0x7f8a78159090 2022-11-27T16:15:10.530 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780b3000 con 0x7f8a7806c570 2022-11-27T16:15:10.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a78159460 0x7f8a78149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:10.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 --2- >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a78159460 0x7f8a78149cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.164:3300/0 says I am v2:172.21.15.164:36978/0 (socket says 172.21.15.164:36978) 2022-11-27T16:15:10.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 -- 172.21.15.164:0/3457931328 learned_addr learned my addr 172.21.15.164:0/3457931328 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:10.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a77fff700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7806c570 0x7f8a780c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:10.532 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a777fe700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78159090 0x7f8a781f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:10.532 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 -- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7806c570 msgr2=0x7f8a780c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:10.532 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7806c570 0x7f8a780c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.532 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 -- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78159090 msgr2=0x7f8a781f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:10.533 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78159090 0x7f8a781f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:10.533 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 -- 172.21.15.164:0/3457931328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a70009b30 con 0x7f8a78159460 2022-11-27T16:15:10.533 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7cabf700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a78159460 0x7f8a78149cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a780715a0 tx=0x7f8a70004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:10.533 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a74ff9700 1 -- 172.21.15.164:0/3457931328 <== mon.0 v2:172.21.15.164:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a700099b0 con 0x7f8a78159460 2022-11-27T16:15:10.534 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a74ff9700 1 -- 172.21.15.164:0/3457931328 <== mon.0 v2:172.21.15.164:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a700092e0 con 0x7f8a78159460 2022-11-27T16:15:10.534 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a74ff9700 1 -- 172.21.15.164:0/3457931328 <== mon.0 v2:172.21.15.164:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7001c640 con 0x7f8a78159460 2022-11-27T16:15:10.534 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a780b32e0 con 0x7f8a78159460 2022-11-27T16:15:10.534 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.520+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a780c5aa0 con 0x7f8a78159460 2022-11-27T16:15:10.535 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.521+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a7806af00 con 0x7f8a78159460 2022-11-27T16:15:10.535 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.522+0000 7f8a74ff9700 1 -- 172.21.15.164:0/3457931328 <== mon.0 v2:172.21.15.164:3300/0 4 ==== mgrmap(e 5) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f8a70022030 con 0x7f8a78159460 2022-11-27T16:15:10.535 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.522+0000 7f8a74ff9700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f8a58066590 0x7f8a58068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:10.535 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.522+0000 7f8a77fff700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f8a58066590 0x7f8a58068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:10.535 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.522+0000 7f8a74ff9700 1 -- 172.21.15.164:0/3457931328 <== mon.0 v2:172.21.15.164:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f8a70084e80 con 0x7f8a78159460 2022-11-27T16:15:10.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.523+0000 7f8a77fff700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f8a58066590 0x7f8a58068a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009930 tx=0x7f8a6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:10.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.527+0000 7f8a74ff9700 1 -- 172.21.15.164:0/3457931328 <== mon.0 v2:172.21.15.164:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f8a70052770 con 0x7f8a78159460 2022-11-27T16:15:10.707 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:10.705+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 --> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f8a780b37a0 con 0x7f8a78159460 2022-11-27T16:15:11.373 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.371+0000 7f8a74ff9700 1 -- 172.21.15.164:0/3457931328 <== mon.0 v2:172.21.15.164:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) v1 ==== 171+0+0 (secure 0 0 0) 0x7f8a70055d90 con 0x7f8a78159460 2022-11-27T16:15:11.378 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.376+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f8a58066590 msgr2=0x7f8a58068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:11.378 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.376+0000 7f8a7e522700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f8a58066590 0x7f8a58068a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8a6c009930 tx=0x7f8a6c008040 comp rx=0 tx=0).stop 2022-11-27T16:15:11.378 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.376+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a78159460 msgr2=0x7f8a78149cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:11.379 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.376+0000 7f8a7e522700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a78159460 0x7f8a78149cf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a780715a0 tx=0x7f8a70004ad0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.379 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.378+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 shutdown_connections 2022-11-27T16:15:11.380 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.378+0000 7f8a7e522700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:6832/101095,v1:172.21.15.164:6833/101095] conn(0x7f8a58066590 0x7f8a58068a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.380 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.378+0000 7f8a7e522700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.164:3300/0,v1:172.21.15.164:6789/0] conn(0x7f8a78159460 0x7f8a78149cf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.380 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.378+0000 7f8a7e522700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f8a7806c570 0x7f8a780c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.380 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.378+0000 7f8a7e522700 1 --2- 172.21.15.164:0/3457931328 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f8a78159090 0x7f8a781f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.381 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.378+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 >> 172.21.15.164:0/3457931328 conn(0x7f8a780cbb40 msgr2=0x7f8a78159970 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:11.381 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.380+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 shutdown_connections 2022-11-27T16:15:11.382 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.380+0000 7f8a7e522700 1 -- 172.21.15.164:0/3457931328 wait complete. 2022-11-27T16:15:11.390 INFO:teuthology.orchestra.run.smithi164.stderr:pool 'cephfs_data' created 2022-11-27T16:15:11.402 DEBUG:teuthology.orchestra.run.smithi164:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-11-27T16:15:11.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.529+0000 7f2e6198b700 1 Processor -- start 2022-11-27T16:15:11.531 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.529+0000 7f2e6198b700 1 -- start start 2022-11-27T16:15:11.534 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e6198b700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e5c06b2f0 0x7f2e5c0ce0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:11.535 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e6198b700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e5c0b5f10 0x7f2e5c0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:15:11.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e6198b700 1 -- --> v1:172.21.15.167:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c0cc6a0 con 0x7f2e5c15b700 2022-11-27T16:15:11.536 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e6198b700 1 -- --> v2:172.21.15.167:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c0ccc60 con 0x7f2e5c0b5f10 2022-11-27T16:15:11.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e6198b700 1 -- --> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0cc980 con 0x7f2e5c06b2f0 2022-11-27T16:15:11.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e5a7fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e5c06b2f0 0x7f2e5c0ce0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:11.537 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e5b7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e5c0b5f10 0x7f2e5c0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:15:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e5a7fc700 1 --2- >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e5c06b2f0 0x7f2e5c0ce0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3301/0 says I am v2:172.21.15.164:51688/0 (socket says 172.21.15.164:51688) 2022-11-27T16:15:11.538 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e5b7fe700 1 --2- >> v2:172.21.15.167:3300/0 conn(0x7f2e5c0b5f10 0x7f2e5c0ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.167:3300/0 says I am v2:172.21.15.164:38224/0 (socket says 172.21.15.164:38224) 2022-11-27T16:15:11.539 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.530+0000 7f2e5a7fc700 1 -- 172.21.15.164:0/3814358736 learned_addr learned my addr 172.21.15.164:0/3814358736 (peer_addr_for_me v2:172.21.15.164:0/0) 2022-11-27T16:15:11.539 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e5b7fe700 1 -- 172.21.15.164:0/3814358736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e5c06b2f0 msgr2=0x7f2e5c0ce0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:15:11.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e5b7fe700 1 --2- 172.21.15.164:0/3814358736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e5c06b2f0 0x7f2e5c0ce0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e5b7fe700 1 -- 172.21.15.164:0/3814358736 >> v1:172.21.15.167:6789/0 conn(0x7f2e5c15b700 legacy=0x7f2e5c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:11.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e59ffb700 1 -- 172.21.15.164:0/3814358736 <== mon.1 v1:172.21.15.167:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1457192440 0 0) 0x7f2e5c0cc6a0 con 0x7f2e5c15b700 2022-11-27T16:15:11.540 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e5b7fe700 1 -- 172.21.15.164:0/3814358736 --> v2:172.21.15.167:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c0d06c0 con 0x7f2e5c0b5f10 2022-11-27T16:15:11.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e5b7fe700 1 --2- 172.21.15.164:0/3814358736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e5c0b5f10 0x7f2e5c0ce630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009d90 tx=0x7f2e5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b548089e434860c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:15:11.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e59ffb700 1 -- 172.21.15.164:0/3814358736 <== mon.1 v2:172.21.15.167:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400e070 con 0x7f2e5c0b5f10 2022-11-27T16:15:11.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e59ffb700 1 -- 172.21.15.164:0/3814358736 <== mon.1 v2:172.21.15.167:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e5400bb20 con 0x7f2e5c0b5f10 2022-11-27T16:15:11.541 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.531+0000 7f2e59ffb700 1 -- 172.21.15.164:0/3814358736 <== mon.1 v2:172.21.15.167:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54012430 con 0x7f2e5c0b5f10 2022-11-27T16:15:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 -- 172.21.15.164:0/3814358736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e5c0b5f10 msgr2=0x7f2e5c0ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:15:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 --2- 172.21.15.164:0/3814358736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e5c0b5f10 0x7f2e5c0ce630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009d90 tx=0x7f2e5400b330 comp rx=0 tx=0).stop 2022-11-27T16:15:11.542 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 -- 172.21.15.164:0/3814358736 shutdown_connections 2022-11-27T16:15:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 --2- 172.21.15.164:0/3814358736 >> [v2:172.21.15.167:3300/0,v1:172.21.15.167:6789/0] conn(0x7f2e5c0b5f10 0x7f2e5c0ce630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.543 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 --2- 172.21.15.164:0/3814358736 >> [v2:172.21.15.167:3301/0,v1:172.21.15.167:6790/0] conn(0x7f2e5c06b2f0 0x7f2e5c0ce0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:15:11.544 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 -- 172.21.15.164:0/3814358736 >> 172.21.15.164:0/3814358736 conn(0x7f2e5c070b00 msgr2=0x7f2e5c070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:15:11.545 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 -- 172.21.15.164:0/3814358736 shutdown_connections 2022-11-27T16:15:11.545 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.532+0000 7f2e6198b700 1 -- 172.21.15.164:0/3814358736 wait complete. 2022-11-27T16:15:11.545 INFO:teuthology.orchestra.run.smithi164.stderr:2022-11-27T16:15:11.533+0000 7f2e6198b700 1 Processor -- start 2022-11-27T16:15:11.546 INFO:teuthology.orchestra.run.smithi1