2022-11-27T16:35:56.944 INFO:root:teuthology version: 0.0.1.dev3+g4da97cf 2022-11-27T16:35:56.947 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --archive /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088670 --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088670/orig.config.yaml 2022-11-27T16:35:57.015 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2022-11-27T16:35:57.170 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088670 branch: main description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: ceph-qa@ceph.io first_in_suite: false job_id: '7088670' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 sleep_before_teardown: 0 suite: fs suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi145.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBADicDbkWTVsl36h8yPnxOjPpkZf/nawuzP7dcV4oZRM/tv9ZZtofvr3r2c1FfJzaNfs+5SvEvk7FI4m6D/h61g= smithi201.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1x/o0GmT7+k5M2rxYh1XiqiIlBd1hAaY6pwKvVnXg7SkCMMzy1Y9/OPMz1gFSTk9iEJXvl/eICGLjSs67ZLcSFvltC+ztPFDFMQFtoibLRbppON8RvKGGIYGurq70wEZWufMFI4e2iVTL5Ha38bxNGAGY8fRbAuFFQPVqDxXHo6IFYnIkRA0fztraW/Ic9sWeDyZdAbUmm4kR83SD07BFx3Noj0XGO1/uxCvfASolmH7nnhldWSLWblZ0puTmPLDDAMLVAGpO1438MecwP4kPcJx+UJxLpHYaZZEhX0bD95p4fWqLV1uueZT6FGhAQFMUn9krUCrw6Z9EicYYHnXpMH4iSlvZgMtr9+z+oj6ygpCHD5JlKTZyZfzoWJzPQ0m2AGZs0nfvFgJ2DEynNBSgQhTrcqncnPekC/9tkH63DdaN/Vl0D3sFo+yJda36HnCco323xXZcQMGKxoUCTFpNPI1ycv9EcgsTvENnzlaU5ISJjcme8hOqNrxEHkamg38= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 4da97cf64e542f347ec47b7bdbe5eca99759f9b7 timestamp: 2022-11-23_17:24:26 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.20399 2022-11-27T16:35:57.171 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa; will attempt to use it 2022-11-27T16:35:57.172 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa/tasks 2022-11-27T16:35:57.172 INFO:teuthology.run_tasks:Running task internal.check_packages... 2022-11-27T16:35:57.174 INFO:teuthology.task.internal:Checking packages... 2022-11-27T16:35:57.204 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '697b0d99d9a52df92b20c8e053dcd46c75f364a9' 2022-11-27T16:35:57.205 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2022-11-27T16:35:57.205 INFO:teuthology.packaging:ref: None 2022-11-27T16:35:57.206 INFO:teuthology.packaging:tag: None 2022-11-27T16:35:57.206 INFO:teuthology.packaging:branch: main 2022-11-27T16:35:57.206 INFO:teuthology.packaging:sha1: 697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:35:57.207 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=main 2022-11-27T16:35:57.441 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-995.gaf6ed506 2022-11-27T16:35:57.441 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2022-11-27T16:35:57.446 INFO:teuthology.task.internal:no buildpackages task found 2022-11-27T16:35:57.447 INFO:teuthology.run_tasks:Running task internal.save_config... 2022-11-27T16:35:57.452 INFO:teuthology.task.internal:Saving configuration 2022-11-27T16:35:57.471 INFO:teuthology.run_tasks:Running task internal.check_lock... 2022-11-27T16:35:57.477 INFO:teuthology.task.internal.check_lock:Checking locks... 2022-11-27T16:35:57.502 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi145.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 16:30:50.365823', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBADicDbkWTVsl36h8yPnxOjPpkZf/nawuzP7dcV4oZRM/tv9ZZtofvr3r2c1FfJzaNfs+5SvEvk7FI4m6D/h61g='} 2022-11-27T16:35:57.527 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi201.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088670', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2022-11-27 16:30:50.364324', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKKQI5NNQUTyWD3n2mriplRrBX6RtkyWf6cj4eiFVLVmfGBYqD1iSXlaTwkhEgyei8n4p3goTKfYVDnTg9tRia4='} 2022-11-27T16:35:57.527 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2022-11-27T16:35:57.535 INFO:teuthology.task.internal:roles: ubuntu@smithi145.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2022-11-27T16:35:57.535 INFO:teuthology.task.internal:roles: ubuntu@smithi201.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2022-11-27T16:35:57.535 INFO:teuthology.run_tasks:Running task console_log... 2022-11-27T16:35:57.612 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2ad6734840>, signals=[15]) 2022-11-27T16:35:57.614 INFO:teuthology.run_tasks:Running task internal.connect... 2022-11-27T16:35:57.622 INFO:teuthology.task.internal:Opening connections... 2022-11-27T16:35:57.623 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi145.front.sepia.ceph.com 2022-11-27T16:35:57.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:35:57.719 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi201.front.sepia.ceph.com 2022-11-27T16:35:57.721 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi201.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:35:57.801 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2022-11-27T16:35:57.808 DEBUG:teuthology.orchestra.run.smithi145:> uname -m 2022-11-27T16:35:57.830 INFO:teuthology.orchestra.run.smithi145.stdout:x86_64 2022-11-27T16:35:57.831 DEBUG:teuthology.orchestra.run.smithi145:> cat /etc/os-release 2022-11-27T16:35:57.890 INFO:teuthology.orchestra.run.smithi145.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T16:35:57.891 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T16:35:57.891 INFO:teuthology.orchestra.run.smithi145.stdout:ID="rhel" 2022-11-27T16:35:57.891 INFO:teuthology.orchestra.run.smithi145.stdout:ID_LIKE="fedora" 2022-11-27T16:35:57.892 INFO:teuthology.orchestra.run.smithi145.stdout:VERSION_ID="8.6" 2022-11-27T16:35:57.892 INFO:teuthology.orchestra.run.smithi145.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:35:57.892 INFO:teuthology.orchestra.run.smithi145.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T16:35:57.892 INFO:teuthology.orchestra.run.smithi145.stdout:ANSI_COLOR="0;31" 2022-11-27T16:35:57.893 INFO:teuthology.orchestra.run.smithi145.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T16:35:57.893 INFO:teuthology.orchestra.run.smithi145.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T16:35:57.893 INFO:teuthology.orchestra.run.smithi145.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T16:35:57.893 INFO:teuthology.orchestra.run.smithi145.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:35:57.894 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:35:57.894 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:35:57.894 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T16:35:57.894 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T16:35:57.895 INFO:teuthology.orchestra.run.smithi145.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T16:35:57.896 INFO:teuthology.lock.ops:Updating smithi145.front.sepia.ceph.com on lock server 2022-11-27T16:35:57.915 DEBUG:teuthology.orchestra.run.smithi201:> uname -m 2022-11-27T16:35:57.936 INFO:teuthology.orchestra.run.smithi201.stdout:x86_64 2022-11-27T16:35:57.936 DEBUG:teuthology.orchestra.run.smithi201:> cat /etc/os-release 2022-11-27T16:35:57.997 INFO:teuthology.orchestra.run.smithi201.stdout:NAME="Red Hat Enterprise Linux" 2022-11-27T16:35:57.998 INFO:teuthology.orchestra.run.smithi201.stdout:VERSION="8.6 (Ootpa)" 2022-11-27T16:35:57.998 INFO:teuthology.orchestra.run.smithi201.stdout:ID="rhel" 2022-11-27T16:35:57.998 INFO:teuthology.orchestra.run.smithi201.stdout:ID_LIKE="fedora" 2022-11-27T16:35:57.999 INFO:teuthology.orchestra.run.smithi201.stdout:VERSION_ID="8.6" 2022-11-27T16:35:57.999 INFO:teuthology.orchestra.run.smithi201.stdout:PLATFORM_ID="platform:el8" 2022-11-27T16:35:57.999 INFO:teuthology.orchestra.run.smithi201.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2022-11-27T16:35:58.000 INFO:teuthology.orchestra.run.smithi201.stdout:ANSI_COLOR="0;31" 2022-11-27T16:35:58.000 INFO:teuthology.orchestra.run.smithi201.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2022-11-27T16:35:58.000 INFO:teuthology.orchestra.run.smithi201.stdout:HOME_URL="https://www.redhat.com/" 2022-11-27T16:35:58.001 INFO:teuthology.orchestra.run.smithi201.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2022-11-27T16:35:58.001 INFO:teuthology.orchestra.run.smithi201.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2022-11-27T16:35:58.001 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:35:58.002 INFO:teuthology.orchestra.run.smithi201.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2022-11-27T16:35:58.002 INFO:teuthology.orchestra.run.smithi201.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2022-11-27T16:35:58.002 INFO:teuthology.orchestra.run.smithi201.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2022-11-27T16:35:58.003 INFO:teuthology.orchestra.run.smithi201.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2022-11-27T16:35:58.003 INFO:teuthology.lock.ops:Updating smithi201.front.sepia.ceph.com on lock server 2022-11-27T16:35:58.024 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2022-11-27T16:35:58.034 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2022-11-27T16:35:58.040 INFO:teuthology.task.internal:Checking for old test directory... 2022-11-27T16:35:58.041 DEBUG:teuthology.orchestra.run.smithi145:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:35:58.043 DEBUG:teuthology.orchestra.run.smithi201:> test '!' -e /home/ubuntu/cephtest 2022-11-27T16:35:58.062 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2022-11-27T16:35:58.073 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2022-11-27T16:35:58.074 DEBUG:teuthology.orchestra.run.smithi145:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:35:58.103 DEBUG:teuthology.orchestra.run.smithi201:> test -z $(ls -A /var/lib/ceph) 2022-11-27T16:35:58.135 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2022-11-27T16:35:58.192 INFO:teuthology.run_tasks:Running task kernel... 2022-11-27T16:35:58.207 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2022-11-27T16:35:58.208 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2022-11-27T16:35:58.208 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2022-11-27T16:35:58.209 DEBUG:teuthology.orchestra.run.smithi145:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:35:58.210 DEBUG:teuthology.orchestra.run.smithi201:> test -f /run/.containerenv -o -f /.dockerenv 2022-11-27T16:35:58.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:35:58.230 DEBUG:teuthology.orchestra.run.smithi145:> uname -r 2022-11-27T16:35:58.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:35:58.232 DEBUG:teuthology.orchestra.run.smithi201:> uname -r 2022-11-27T16:35:58.289 INFO:teuthology.orchestra.run.smithi145.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T16:35:58.290 INFO:teuthology.task.kernel:Running kernel on smithi145: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:35:58.290 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum install -y kernel 2022-11-27T16:35:58.291 INFO:teuthology.orchestra.run.smithi201.stdout:4.18.0-372.9.1.el8.x86_64 2022-11-27T16:35:58.292 INFO:teuthology.task.kernel:Running kernel on smithi201: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:35:58.292 DEBUG:teuthology.orchestra.run.smithi201:> sudo yum install -y kernel 2022-11-27T16:35:59.281 INFO:teuthology.orchestra.run.smithi201.stdout:Updating Subscription Management repositories. 2022-11-27T16:35:59.281 INFO:teuthology.orchestra.run.smithi201.stdout:Unable to read consumer identity 2022-11-27T16:35:59.325 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2022-11-27T16:35:59.326 INFO:teuthology.orchestra.run.smithi145.stdout:Unable to read consumer identity 2022-11-27T16:35:59.378 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:35:59.379 INFO:teuthology.orchestra.run.smithi201.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:35:59.379 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:35:59.404 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:35:59.404 INFO:teuthology.orchestra.run.smithi145.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:35:59.405 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:36:02.937 INFO:teuthology.orchestra.run.smithi201.stdout:Copr repo for python3-asyncssh owned by ceph 1.4 kB/s | 3.5 kB 00:02 2022-11-27T16:36:03.280 INFO:teuthology.orchestra.run.smithi145.stdout:Copr repo for python3-asyncssh owned by ceph 1.2 kB/s | 3.5 kB 00:02 2022-11-27T16:36:03.961 INFO:teuthology.orchestra.run.smithi201.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2022-11-27T16:36:04.339 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 13 MB/s | 13 MB 00:01 2022-11-27T16:36:06.895 INFO:teuthology.orchestra.run.smithi201.stdout:lab-extras 640 kB/s | 24 kB 00:00 2022-11-27T16:36:07.280 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 458 kB/s | 24 kB 00:00 2022-11-27T16:36:07.564 INFO:teuthology.orchestra.run.smithi201.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T16:36:07.615 INFO:teuthology.orchestra.run.smithi201.stdout:Dependencies resolved. 2022-11-27T16:36:07.616 INFO:teuthology.orchestra.run.smithi201.stdout:Nothing to do. 2022-11-27T16:36:07.617 INFO:teuthology.orchestra.run.smithi201.stdout:Complete! 2022-11-27T16:36:07.751 DEBUG:teuthology.orchestra.run.smithi201:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:36:07.949 INFO:teuthology.orchestra.run.smithi145.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2022-11-27T16:36:08.002 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2022-11-27T16:36:08.003 INFO:teuthology.orchestra.run.smithi145.stdout:Nothing to do. 2022-11-27T16:36:08.003 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2022-11-27T16:36:08.121 INFO:teuthology.orchestra.run.smithi201.stdout:Updating Subscription Management repositories. 2022-11-27T16:36:08.121 INFO:teuthology.orchestra.run.smithi201.stdout:Unable to read consumer identity 2022-11-27T16:36:08.134 DEBUG:teuthology.orchestra.run.smithi145:> echo no | sudo yum reinstall kernel || true 2022-11-27T16:36:08.171 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:36:08.171 INFO:teuthology.orchestra.run.smithi201.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:36:08.171 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:36:08.417 INFO:teuthology.orchestra.run.smithi201.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 04:36:06 PM UTC. 2022-11-27T16:36:08.464 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2022-11-27T16:36:08.465 INFO:teuthology.orchestra.run.smithi145.stdout:Unable to read consumer identity 2022-11-27T16:36:08.478 INFO:teuthology.orchestra.run.smithi201.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:36:08.508 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:36:08.508 INFO:teuthology.orchestra.run.smithi145.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:36:08.508 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:36:08.511 INFO:teuthology.orchestra.run.smithi201.stderr:Error: No packages marked for reinstall. 2022-11-27T16:36:08.558 DEBUG:teuthology.orchestra.run.smithi201:> sudo yum reinstall -y kernel || true 2022-11-27T16:36:08.746 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:01 ago on Sun 27 Nov 2022 04:36:07 PM UTC. 2022-11-27T16:36:08.786 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:36:08.829 INFO:teuthology.orchestra.run.smithi145.stderr:Error: No packages marked for reinstall. 2022-11-27T16:36:08.876 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum reinstall -y kernel || true 2022-11-27T16:36:08.887 INFO:teuthology.orchestra.run.smithi201.stdout:Updating Subscription Management repositories. 2022-11-27T16:36:08.888 INFO:teuthology.orchestra.run.smithi201.stdout:Unable to read consumer identity 2022-11-27T16:36:08.939 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:36:08.939 INFO:teuthology.orchestra.run.smithi201.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:36:08.939 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:36:09.175 INFO:teuthology.orchestra.run.smithi201.stdout:Last metadata expiration check: 0:00:03 ago on Sun 27 Nov 2022 04:36:06 PM UTC. 2022-11-27T16:36:09.205 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2022-11-27T16:36:09.205 INFO:teuthology.orchestra.run.smithi145.stdout:Unable to read consumer identity 2022-11-27T16:36:09.216 INFO:teuthology.orchestra.run.smithi201.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:36:09.241 INFO:teuthology.orchestra.run.smithi201.stderr:Error: No packages marked for reinstall. 2022-11-27T16:36:09.259 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:36:09.259 INFO:teuthology.orchestra.run.smithi145.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2022-11-27T16:36:09.260 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:36:09.289 DEBUG:teuthology.orchestra.run.smithi201:> rpm -q kernel --last | head -n 1 2022-11-27T16:36:09.369 INFO:teuthology.orchestra.run.smithi201.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T16:36:09.370 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:36:09.371 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:36:09.371 DEBUG:teuthology.parallel:result is None 2022-11-27T16:36:09.494 INFO:teuthology.orchestra.run.smithi145.stdout:Last metadata expiration check: 0:00:02 ago on Sun 27 Nov 2022 04:36:07 PM UTC. 2022-11-27T16:36:09.532 INFO:teuthology.orchestra.run.smithi145.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2022-11-27T16:36:09.561 INFO:teuthology.orchestra.run.smithi145.stderr:Error: No packages marked for reinstall. 2022-11-27T16:36:09.608 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q kernel --last | head -n 1 2022-11-27T16:36:09.689 INFO:teuthology.orchestra.run.smithi145.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2022-11-27T16:36:09.691 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2022-11-27T16:36:09.691 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2022-11-27T16:36:09.691 DEBUG:teuthology.parallel:result is None 2022-11-27T16:36:09.692 INFO:teuthology.run_tasks:Running task internal.base... 2022-11-27T16:36:09.699 INFO:teuthology.task.internal:Creating test directory... 2022-11-27T16:36:09.699 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:36:09.702 DEBUG:teuthology.orchestra.run.smithi201:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2022-11-27T16:36:09.719 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2022-11-27T16:36:09.726 INFO:teuthology.run_tasks:Running task internal.archive... 2022-11-27T16:36:09.736 INFO:teuthology.task.internal:Creating archive directory... 2022-11-27T16:36:09.736 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:36:09.759 DEBUG:teuthology.orchestra.run.smithi201:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2022-11-27T16:36:09.791 INFO:teuthology.run_tasks:Running task internal.coredump... 2022-11-27T16:36:09.799 INFO:teuthology.task.internal:Enabling coredump saving... 2022-11-27T16:36:09.800 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:36:09.828 DEBUG:teuthology.orchestra.run.smithi201:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2022-11-27T16:36:09.857 INFO:teuthology.orchestra.run.smithi201.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:36:09.862 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:36:09.875 INFO:teuthology.orchestra.run.smithi201.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:36:09.877 INFO:teuthology.orchestra.run.smithi145.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2022-11-27T16:36:09.877 INFO:teuthology.run_tasks:Running task internal.sudo... 2022-11-27T16:36:09.885 INFO:teuthology.task.internal:Configuring sudo... 2022-11-27T16:36:09.885 DEBUG:teuthology.orchestra.run.smithi145:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:36:09.920 DEBUG:teuthology.orchestra.run.smithi201:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2022-11-27T16:36:09.947 INFO:teuthology.run_tasks:Running task internal.syslog... 2022-11-27T16:36:09.960 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2022-11-27T16:36:09.960 DEBUG:teuthology.orchestra.run.smithi145:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:36:09.988 DEBUG:teuthology.orchestra.run.smithi201:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2022-11-27T16:36:10.004 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:36:10.085 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:36:10.134 DEBUG:teuthology.orchestra.run.smithi145:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:36:10.220 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:36:10.260 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:36:10.261 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:36:10.327 DEBUG:teuthology.orchestra.run.smithi201:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:36:10.370 DEBUG:teuthology.orchestra.run.smithi201:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2022-11-27T16:36:10.443 DEBUG:teuthology.orchestra.run.smithi201:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:36:10.489 DEBUG:teuthology.orchestra.run.smithi201:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2022-11-27T16:36:10.529 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:36:10.529 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2022-11-27T16:36:10.595 DEBUG:teuthology.orchestra.run.smithi145:> sudo service rsyslog restart 2022-11-27T16:36:10.598 DEBUG:teuthology.orchestra.run.smithi201:> sudo service rsyslog restart 2022-11-27T16:36:10.650 INFO:teuthology.orchestra.run.smithi145.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:36:10.677 INFO:teuthology.orchestra.run.smithi201.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2022-11-27T16:36:11.008 INFO:teuthology.run_tasks:Running task internal.timer... 2022-11-27T16:36:11.017 INFO:teuthology.task.internal:Starting timer... 2022-11-27T16:36:11.017 INFO:teuthology.run_tasks:Running task pcp... 2022-11-27T16:36:11.028 INFO:teuthology.run_tasks:Running task selinux... 2022-11-27T16:36:11.040 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2022-11-27T16:36:11.041 DEBUG:teuthology.orchestra.run.smithi145:> sudo service auditd rotate 2022-11-27T16:36:11.095 INFO:teuthology.orchestra.run.smithi145.stdout:Rotating logs: 2022-11-27T16:36:11.097 DEBUG:teuthology.orchestra.run.smithi201:> sudo service auditd rotate 2022-11-27T16:36:11.161 INFO:teuthology.orchestra.run.smithi201.stdout:Rotating logs: 2022-11-27T16:36:11.164 DEBUG:teuthology.task.selinux:Getting current SELinux state 2022-11-27T16:36:11.165 DEBUG:teuthology.orchestra.run.smithi145:> /usr/sbin/getenforce 2022-11-27T16:36:11.187 INFO:teuthology.orchestra.run.smithi145.stdout:Permissive 2022-11-27T16:36:11.188 DEBUG:teuthology.orchestra.run.smithi201:> /usr/sbin/getenforce 2022-11-27T16:36:11.216 INFO:teuthology.orchestra.run.smithi201.stdout:Permissive 2022-11-27T16:36:11.216 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi145.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi201.front.sepia.ceph.com': 'permissive'} 2022-11-27T16:36:11.217 DEBUG:teuthology.orchestra.run.smithi145:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:36:11.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:36:11.256 DEBUG:teuthology.orchestra.run.smithi201:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2022-11-27T16:36:11.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2022-11-27T16:36:11.285 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2022-11-27T16:36:11.286 DEBUG:teuthology.orchestra.run.smithi145:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:36:11.323 DEBUG:teuthology.orchestra.run.smithi201:> sudo /usr/sbin/setenforce permissive 2022-11-27T16:36:11.355 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2022-11-27T16:36:11.367 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2022-11-27T16:36:11.374 INFO:teuthology.repo_utils:Fetching main from origin 2022-11-27T16:36:11.527 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2022-11-27T16:36:11.550 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2022-11-27T16:36:11.552 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi145.front.sepia.ceph.com,smithi201.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2022-11-27T16:44:04.685 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi145.front.sepia.ceph.com'), Remote(name='ubuntu@smithi201.front.sepia.ceph.com')] 2022-11-27T16:44:04.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi145.front.sepia.ceph.com' 2022-11-27T16:44:04.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi145.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:44:04.774 DEBUG:teuthology.orchestra.run.smithi145:> true 2022-11-27T16:44:04.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi201.front.sepia.ceph.com' 2022-11-27T16:44:04.843 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi201.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2022-11-27T16:44:04.926 DEBUG:teuthology.orchestra.run.smithi201:> true 2022-11-27T16:44:04.993 INFO:teuthology.run_tasks:Running task clock... 2022-11-27T16:44:05.007 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2022-11-27T16:44:05.008 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:44:05.013 DEBUG:teuthology.orchestra.run.smithi145:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:44:05.017 INFO:teuthology.orchestra.run:Running command with timeout 360 2022-11-27T16:44:05.017 DEBUG:teuthology.orchestra.run.smithi201:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2022-11-27T16:44:05.049 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:44:05.064 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:44:05.065 INFO:teuthology.orchestra.run.smithi201.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2022-11-27T16:44:05.080 INFO:teuthology.orchestra.run.smithi201.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2022-11-27T16:44:05.141 INFO:teuthology.orchestra.run.smithi145.stderr:sudo: ntpd: command not found 2022-11-27T16:44:05.142 INFO:teuthology.orchestra.run.smithi201.stderr:sudo: ntpd: command not found 2022-11-27T16:44:05.153 INFO:teuthology.orchestra.run.smithi145.stdout:506 Cannot talk to daemon 2022-11-27T16:44:05.154 INFO:teuthology.orchestra.run.smithi201.stdout:506 Cannot talk to daemon 2022-11-27T16:44:05.167 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:44:05.168 INFO:teuthology.orchestra.run.smithi201.stderr:Failed to start ntp.service: Unit ntp.service not found. 2022-11-27T16:44:05.181 INFO:teuthology.orchestra.run.smithi145.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:44:05.182 INFO:teuthology.orchestra.run.smithi201.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2022-11-27T16:44:05.251 INFO:teuthology.orchestra.run.smithi201.stderr:bash: ntpq: command not found 2022-11-27T16:44:05.254 INFO:teuthology.orchestra.run.smithi201.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:44:05.255 INFO:teuthology.orchestra.run.smithi201.stdout:=============================================================================== 2022-11-27T16:44:05.255 INFO:teuthology.orchestra.run.smithi201.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.255 INFO:teuthology.orchestra.run.smithi201.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.255 INFO:teuthology.orchestra.run.smithi201.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.256 INFO:teuthology.orchestra.run.smithi201.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.263 INFO:teuthology.orchestra.run.smithi145.stderr:bash: ntpq: command not found 2022-11-27T16:44:05.267 INFO:teuthology.orchestra.run.smithi145.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2022-11-27T16:44:05.268 INFO:teuthology.orchestra.run.smithi145.stdout:=============================================================================== 2022-11-27T16:44:05.269 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.270 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.270 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.270 INFO:teuthology.orchestra.run.smithi145.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2022-11-27T16:44:05.271 INFO:teuthology.run_tasks:Running task install... 2022-11-27T16:44:05.281 DEBUG:teuthology.task.install:project ceph 2022-11-27T16:44:05.282 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}} 2022-11-27T16:44:05.282 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'} 2022-11-27T16:44:05.283 INFO:teuthology.task.install:Using flavor: default 2022-11-27T16:44:05.293 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2022-11-27T16:44:05.294 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2022-11-27T16:44:05.295 DEBUG:teuthology.orchestra.run.smithi145:> sudo lsb_release -is 2022-11-27T16:44:05.349 INFO:teuthology.orchestra.run.smithi145.stdout:RedHatEnterprise 2022-11-27T16:44:05.351 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:44:05.352 DEBUG:teuthology.orchestra.run.smithi201:> sudo lsb_release -is 2022-11-27T16:44:05.353 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:44:05.354 DEBUG:teuthology.orchestra.run.smithi145:> sudo lsb_release -is 2022-11-27T16:44:05.395 INFO:teuthology.orchestra.run.smithi201.stdout:RedHatEnterprise 2022-11-27T16:44:05.396 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:44:05.397 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'tag': None, 'wait_for_package': False} 2022-11-27T16:44:05.397 DEBUG:teuthology.orchestra.run.smithi201:> sudo lsb_release -is 2022-11-27T16:44:05.432 INFO:teuthology.orchestra.run.smithi145.stdout:RedHatEnterprise 2022-11-27T16:44:05.433 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:44:05.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:44:05.479 INFO:teuthology.orchestra.run.smithi201.stdout:RedHatEnterprise 2022-11-27T16:44:05.480 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:44:05.481 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:44:05.594 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:44:05.594 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:44:05.638 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/ 2022-11-27T16:44:05.638 INFO:teuthology.task.install.rpm:Package version is 18.0.0-594.g697b0d99 2022-11-27T16:44:05.716 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:44:05.716 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:44:05.717 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:44:05.744 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:44:05.744 DEBUG:teuthology.orchestra.run.smithi145:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:44:05.759 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/main/697b0d99d9a52df92b20c8e053dcd46c75f364a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2022-11-27T16:44:05.760 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:44:05.761 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd of=/etc/yum.repos.d/ceph.repo 2022-11-27T16:44:05.788 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2022-11-27T16:44:05.788 DEBUG:teuthology.orchestra.run.smithi201:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/697b0d99d9a52df92b20c8e053dcd46c75f364a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2022-11-27T16:44:05.812 DEBUG:teuthology.orchestra.run.smithi145:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:44:05.856 DEBUG:teuthology.orchestra.run.smithi201:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2022-11-27T16:44:05.889 DEBUG:teuthology.orchestra.run.smithi145:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:44:05.933 DEBUG:teuthology.orchestra.run.smithi201:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2022-11-27T16:44:05.956 INFO:teuthology.orchestra.run.smithi145.stdout:check_obsoletes = 1 2022-11-27T16:44:05.957 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum clean all 2022-11-27T16:44:05.999 INFO:teuthology.orchestra.run.smithi201.stdout:check_obsoletes = 1 2022-11-27T16:44:06.002 DEBUG:teuthology.orchestra.run.smithi201:> sudo yum clean all 2022-11-27T16:44:06.328 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2022-11-27T16:44:06.373 INFO:teuthology.orchestra.run.smithi201.stdout:Updating Subscription Management repositories. 2022-11-27T16:44:06.530 INFO:teuthology.orchestra.run.smithi145.stdout:54 files removed 2022-11-27T16:44:06.577 DEBUG:teuthology.orchestra.run.smithi145:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:44:06.578 INFO:teuthology.orchestra.run.smithi201.stdout:54 files removed 2022-11-27T16:44:06.626 DEBUG:teuthology.orchestra.run.smithi201:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2022-11-27T16:44:06.907 INFO:teuthology.orchestra.run.smithi145.stdout:Updating Subscription Management repositories. 2022-11-27T16:44:06.958 INFO:teuthology.orchestra.run.smithi201.stdout:Updating Subscription Management repositories. 2022-11-27T16:44:07.510 INFO:teuthology.orchestra.run.smithi145.stdout:ceph packages for x86_64 306 kB/s | 81 kB 00:00 2022-11-27T16:44:07.554 INFO:teuthology.orchestra.run.smithi201.stdout:ceph packages for x86_64 291 kB/s | 81 kB 00:00 2022-11-27T16:44:07.777 INFO:teuthology.orchestra.run.smithi145.stdout:ceph noarch packages 40 kB/s | 9.4 kB 00:00 2022-11-27T16:44:07.830 INFO:teuthology.orchestra.run.smithi201.stdout:ceph noarch packages 38 kB/s | 9.4 kB 00:00 2022-11-27T16:44:08.068 INFO:teuthology.orchestra.run.smithi145.stdout:ceph source packages 5.6 kB/s | 1.5 kB 00:00 2022-11-27T16:44:08.089 INFO:teuthology.orchestra.run.smithi201.stdout:ceph source packages 6.4 kB/s | 1.5 kB 00:00 2022-11-27T16:44:11.681 INFO:teuthology.orchestra.run.smithi201.stdout:Copr repo for python3-asyncssh owned by ceph 993 B/s | 3.5 kB 00:03 2022-11-27T16:44:11.811 INFO:teuthology.orchestra.run.smithi145.stdout:Copr repo for python3-asyncssh owned by ceph 953 B/s | 3.5 kB 00:03 2022-11-27T16:44:15.389 INFO:teuthology.orchestra.run.smithi201.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 13 MB 00:03 2022-11-27T16:44:15.494 INFO:teuthology.orchestra.run.smithi145.stdout:Extra Packages for Enterprise Linux 3.6 MB/s | 13 MB 00:03 2022-11-27T16:44:18.331 INFO:teuthology.orchestra.run.smithi201.stdout:lab-extras 496 kB/s | 24 kB 00:00 2022-11-27T16:44:18.453 INFO:teuthology.orchestra.run.smithi145.stdout:lab-extras 476 kB/s | 24 kB 00:00 2022-11-27T16:44:19.174 INFO:teuthology.orchestra.run.smithi201.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 68 MB/s | 53 MB 00:00 2022-11-27T16:44:19.222 INFO:teuthology.orchestra.run.smithi145.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 68 MB/s | 48 MB 00:00 2022-11-27T16:44:27.192 INFO:teuthology.orchestra.run.smithi201.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 67 MB/s | 48 MB 00:00 2022-11-27T16:44:27.255 INFO:teuthology.orchestra.run.smithi145.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 68 MB/s | 53 MB 00:00 2022-11-27T16:44:38.357 INFO:teuthology.orchestra.run.smithi201.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:44:38.358 INFO:teuthology.orchestra.run.smithi201.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:44:38.373 INFO:teuthology.orchestra.run.smithi201.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T16:44:38.374 INFO:teuthology.orchestra.run.smithi201.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T16:44:38.375 INFO:teuthology.orchestra.run.smithi201.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T16:44:38.376 INFO:teuthology.orchestra.run.smithi201.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:44:38.379 INFO:teuthology.orchestra.run.smithi201.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:44:38.380 INFO:teuthology.orchestra.run.smithi145.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:44:38.381 INFO:teuthology.orchestra.run.smithi145.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2022-11-27T16:44:38.382 INFO:teuthology.orchestra.run.smithi201.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:44:38.383 INFO:teuthology.orchestra.run.smithi201.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:44:38.396 INFO:teuthology.orchestra.run.smithi145.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2022-11-27T16:44:38.397 INFO:teuthology.orchestra.run.smithi145.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2022-11-27T16:44:38.398 INFO:teuthology.orchestra.run.smithi145.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2022-11-27T16:44:38.400 INFO:teuthology.orchestra.run.smithi145.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2022-11-27T16:44:38.402 INFO:teuthology.orchestra.run.smithi145.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2022-11-27T16:44:38.405 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2022-11-27T16:44:38.407 INFO:teuthology.orchestra.run.smithi145.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2022-11-27T16:44:38.514 INFO:teuthology.orchestra.run.smithi201.stdout:Dependencies resolved. 2022-11-27T16:44:38.528 INFO:teuthology.orchestra.run.smithi201.stdout:=========================================================================================================================== 2022-11-27T16:44:38.528 INFO:teuthology.orchestra.run.smithi201.stdout: Package Arch Version Repository Size 2022-11-27T16:44:38.528 INFO:teuthology.orchestra.run.smithi201.stdout:=========================================================================================================================== 2022-11-27T16:44:38.529 INFO:teuthology.orchestra.run.smithi201.stdout:Installing: 2022-11-27T16:44:38.529 INFO:teuthology.orchestra.run.smithi201.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T16:44:38.529 INFO:teuthology.orchestra.run.smithi201.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T16:44:38.529 INFO:teuthology.orchestra.run.smithi201.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:44:38.530 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:44:38.530 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:44:38.530 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:44:38.530 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:44:38.531 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:44:38.531 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:44:38.531 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:44:38.531 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:44:38.532 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:44:38.532 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:44:38.532 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:44:38.532 INFO:teuthology.orchestra.run.smithi201.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:44:38.533 INFO:teuthology.orchestra.run.smithi201.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:44:38.533 INFO:teuthology.orchestra.run.smithi201.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:44:38.533 INFO:teuthology.orchestra.run.smithi201.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T16:44:38.533 INFO:teuthology.orchestra.run.smithi201.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T16:44:38.534 INFO:teuthology.orchestra.run.smithi201.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T16:44:38.534 INFO:teuthology.orchestra.run.smithi201.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:44:38.534 INFO:teuthology.orchestra.run.smithi201.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:44:38.534 INFO:teuthology.orchestra.run.smithi201.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:44:38.534 INFO:teuthology.orchestra.run.smithi201.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T16:44:38.535 INFO:teuthology.orchestra.run.smithi201.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T16:44:38.535 INFO:teuthology.orchestra.run.smithi201.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T16:44:38.535 INFO:teuthology.orchestra.run.smithi201.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T16:44:38.535 INFO:teuthology.orchestra.run.smithi201.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T16:44:38.536 INFO:teuthology.orchestra.run.smithi201.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T16:44:38.536 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:44:38.536 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:44:38.536 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:44:38.537 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:44:38.537 INFO:teuthology.orchestra.run.smithi201.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:44:38.537 INFO:teuthology.orchestra.run.smithi201.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:44:38.537 INFO:teuthology.orchestra.run.smithi201.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:44:38.538 INFO:teuthology.orchestra.run.smithi201.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T16:44:38.538 INFO:teuthology.orchestra.run.smithi201.stdout:Upgrading: 2022-11-27T16:44:38.538 INFO:teuthology.orchestra.run.smithi201.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T16:44:38.538 INFO:teuthology.orchestra.run.smithi201.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:44:38.539 INFO:teuthology.orchestra.run.smithi201.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:44:38.539 INFO:teuthology.orchestra.run.smithi201.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T16:44:38.539 INFO:teuthology.orchestra.run.smithi201.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T16:44:38.539 INFO:teuthology.orchestra.run.smithi201.stdout:Installing dependencies: 2022-11-27T16:44:38.539 INFO:teuthology.orchestra.run.smithi201.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T16:44:38.540 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:44:38.540 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:44:38.540 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:44:38.540 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:44:38.541 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:44:38.541 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:44:38.541 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:44:38.541 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:44:38.542 INFO:teuthology.orchestra.run.smithi201.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T16:44:38.542 INFO:teuthology.orchestra.run.smithi201.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:44:38.542 INFO:teuthology.orchestra.run.smithi201.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T16:44:38.542 INFO:teuthology.orchestra.run.smithi201.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T16:44:38.542 INFO:teuthology.orchestra.run.smithi201.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T16:44:38.543 INFO:teuthology.orchestra.run.smithi201.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:44:38.543 INFO:teuthology.orchestra.run.smithi201.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T16:44:38.543 INFO:teuthology.orchestra.run.smithi201.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T16:44:38.543 INFO:teuthology.orchestra.run.smithi201.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:44:38.544 INFO:teuthology.orchestra.run.smithi201.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T16:44:38.544 INFO:teuthology.orchestra.run.smithi201.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T16:44:38.544 INFO:teuthology.orchestra.run.smithi201.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T16:44:38.544 INFO:teuthology.orchestra.run.smithi201.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:44:38.544 INFO:teuthology.orchestra.run.smithi201.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T16:44:38.545 INFO:teuthology.orchestra.run.smithi201.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:44:38.545 INFO:teuthology.orchestra.run.smithi201.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T16:44:38.545 INFO:teuthology.orchestra.run.smithi201.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T16:44:38.545 INFO:teuthology.orchestra.run.smithi201.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:44:38.546 INFO:teuthology.orchestra.run.smithi201.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T16:44:38.546 INFO:teuthology.orchestra.run.smithi201.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T16:44:38.546 INFO:teuthology.orchestra.run.smithi201.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T16:44:38.546 INFO:teuthology.orchestra.run.smithi201.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T16:44:38.547 INFO:teuthology.orchestra.run.smithi201.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T16:44:38.547 INFO:teuthology.orchestra.run.smithi201.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T16:44:38.547 INFO:teuthology.orchestra.run.smithi201.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T16:44:38.547 INFO:teuthology.orchestra.run.smithi201.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T16:44:38.547 INFO:teuthology.orchestra.run.smithi201.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:44:38.548 INFO:teuthology.orchestra.run.smithi201.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T16:44:38.548 INFO:teuthology.orchestra.run.smithi201.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:44:38.548 INFO:teuthology.orchestra.run.smithi201.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:44:38.548 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:44:38.549 INFO:teuthology.orchestra.run.smithi201.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:44:38.549 INFO:teuthology.orchestra.run.smithi201.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:44:38.549 INFO:teuthology.orchestra.run.smithi201.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:44:38.549 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T16:44:38.550 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:44:38.550 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:44:38.550 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T16:44:38.550 INFO:teuthology.orchestra.run.smithi201.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:44:38.550 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:44:38.551 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:44:38.551 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T16:44:38.551 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T16:44:38.551 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T16:44:38.552 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:44:38.552 INFO:teuthology.orchestra.run.smithi201.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:44:38.552 INFO:teuthology.orchestra.run.smithi201.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:44:38.552 INFO:teuthology.orchestra.run.smithi201.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T16:44:38.552 INFO:teuthology.orchestra.run.smithi201.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T16:44:38.553 INFO:teuthology.orchestra.run.smithi201.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:44:38.553 INFO:teuthology.orchestra.run.smithi201.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:44:38.553 INFO:teuthology.orchestra.run.smithi201.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T16:44:38.553 INFO:teuthology.orchestra.run.smithi201.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T16:44:38.554 INFO:teuthology.orchestra.run.smithi201.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T16:44:38.554 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:44:38.554 INFO:teuthology.orchestra.run.smithi201.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:44:38.554 INFO:teuthology.orchestra.run.smithi201.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T16:44:38.554 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T16:44:38.555 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T16:44:38.555 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T16:44:38.555 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T16:44:38.555 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T16:44:38.556 INFO:teuthology.orchestra.run.smithi201.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:44:38.556 INFO:teuthology.orchestra.run.smithi201.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:44:38.556 INFO:teuthology.orchestra.run.smithi201.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:44:38.556 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:44:38.556 INFO:teuthology.orchestra.run.smithi201.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T16:44:38.557 INFO:teuthology.orchestra.run.smithi201.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:44:38.557 INFO:teuthology.orchestra.run.smithi201.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:44:38.557 INFO:teuthology.orchestra.run.smithi201.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:44:38.557 INFO:teuthology.orchestra.run.smithi201.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:44:38.558 INFO:teuthology.orchestra.run.smithi201.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:44:38.558 INFO:teuthology.orchestra.run.smithi201.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:44:38.558 INFO:teuthology.orchestra.run.smithi201.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:44:38.558 INFO:teuthology.orchestra.run.smithi201.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:44:38.558 INFO:teuthology.orchestra.run.smithi201.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T16:44:38.559 INFO:teuthology.orchestra.run.smithi201.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:44:38.559 INFO:teuthology.orchestra.run.smithi201.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T16:44:38.559 INFO:teuthology.orchestra.run.smithi201.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:44:38.559 INFO:teuthology.orchestra.run.smithi201.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T16:44:38.560 INFO:teuthology.orchestra.run.smithi201.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T16:44:38.560 INFO:teuthology.orchestra.run.smithi201.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T16:44:38.560 INFO:teuthology.orchestra.run.smithi201.stdout:Enabling module streams: 2022-11-27T16:44:38.560 INFO:teuthology.orchestra.run.smithi201.stdout: postgresql 10 2022-11-27T16:44:38.560 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:44:38.561 INFO:teuthology.orchestra.run.smithi201.stdout:Transaction Summary 2022-11-27T16:44:38.561 INFO:teuthology.orchestra.run.smithi201.stdout:=========================================================================================================================== 2022-11-27T16:44:38.561 INFO:teuthology.orchestra.run.smithi201.stdout:Install 128 Packages 2022-11-27T16:44:38.561 INFO:teuthology.orchestra.run.smithi201.stdout:Upgrade 5 Packages 2022-11-27T16:44:38.562 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:44:38.563 INFO:teuthology.orchestra.run.smithi201.stdout:Total download size: 219 M 2022-11-27T16:44:38.563 INFO:teuthology.orchestra.run.smithi201.stdout:Downloading Packages: 2022-11-27T16:44:38.564 INFO:teuthology.orchestra.run.smithi145.stdout:Dependencies resolved. 2022-11-27T16:44:38.564 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================================================== 2022-11-27T16:44:38.564 INFO:teuthology.orchestra.run.smithi145.stdout: Package Arch Version Repository Size 2022-11-27T16:44:38.565 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================================================== 2022-11-27T16:44:38.565 INFO:teuthology.orchestra.run.smithi145.stdout:Installing: 2022-11-27T16:44:38.565 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2022-11-27T16:44:38.565 INFO:teuthology.orchestra.run.smithi145.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2022-11-27T16:44:38.566 INFO:teuthology.orchestra.run.smithi145.stdout: ceph x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.4 k 2022-11-27T16:44:38.566 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base x86_64 2:18.0.0-594.g697b0d99.el8 ceph 5.2 M 2022-11-27T16:44:38.566 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 845 k 2022-11-27T16:44:38.566 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-594.g697b0d99.el8 ceph 165 k 2022-11-27T16:44:38.567 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr x86_64 2:18.0.0-594.g697b0d99.el8 ceph 1.5 M 2022-11-27T16:44:38.567 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 118 k 2022-11-27T16:44:38.567 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 1.7 M 2022-11-27T16:44:38.567 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 7.4 M 2022-11-27T16:44:38.567 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 47 k 2022-11-27T16:44:38.568 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 18 M 2022-11-27T16:44:38.568 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test x86_64 2:18.0.0-594.g697b0d99.el8 ceph 55 M 2022-11-27T16:44:38.568 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 263 k 2022-11-27T16:44:38.568 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 81 k 2022-11-27T16:44:38.569 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 257 k 2022-11-27T16:44:38.569 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 15 k 2022-11-27T16:44:38.569 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2022-11-27T16:44:38.569 INFO:teuthology.orchestra.run.smithi145.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2022-11-27T16:44:38.569 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2022-11-27T16:44:38.570 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 30 k 2022-11-27T16:44:38.570 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 767 k 2022-11-27T16:44:38.570 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel x86_64 2:18.0.0-594.g697b0d99.el8 ceph 139 k 2022-11-27T16:44:38.570 INFO:teuthology.orchestra.run.smithi145.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2022-11-27T16:44:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2022-11-27T16:44:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2022-11-27T16:44:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2022-11-27T16:44:38.571 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2022-11-27T16:44:38.572 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2022-11-27T16:44:38.572 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs x86_64 2:18.0.0-594.g697b0d99.el8 ceph 190 k 2022-11-27T16:44:38.572 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados x86_64 2:18.0.0-594.g697b0d99.el8 ceph 371 k 2022-11-27T16:44:38.572 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 357 k 2022-11-27T16:44:38.572 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw x86_64 2:18.0.0-594.g697b0d99.el8 ceph 107 k 2022-11-27T16:44:38.573 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 94 k 2022-11-27T16:44:38.573 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.5 M 2022-11-27T16:44:38.573 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 175 k 2022-11-27T16:44:38.573 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2022-11-27T16:44:38.574 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrading: 2022-11-27T16:44:38.574 INFO:teuthology.orchestra.run.smithi145.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2022-11-27T16:44:38.574 INFO:teuthology.orchestra.run.smithi145.stdout: librados2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:44:38.574 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.6 M 2022-11-27T16:44:38.574 INFO:teuthology.orchestra.run.smithi145.stdout: openssl x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 709 k 2022-11-27T16:44:38.575 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 1.5 M 2022-11-27T16:44:38.575 INFO:teuthology.orchestra.run.smithi145.stdout:Installing dependencies: 2022-11-27T16:44:38.575 INFO:teuthology.orchestra.run.smithi145.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2022-11-27T16:44:38.575 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 M 2022-11-27T16:44:38.576 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 23 k 2022-11-27T16:44:38.576 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds x86_64 2:18.0.0-594.g697b0d99.el8 ceph 2.3 M 2022-11-27T16:44:38.576 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 234 k 2022-11-27T16:44:38.576 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon x86_64 2:18.0.0-594.g697b0d99.el8 ceph 3.9 M 2022-11-27T16:44:38.576 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd x86_64 2:18.0.0-594.g697b0d99.el8 ceph 17 M 2022-11-27T16:44:38.577 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts noarch 2:18.0.0-594.g697b0d99.el8 ceph-noarch 14 k 2022-11-27T16:44:38.577 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux x86_64 2:18.0.0-594.g697b0d99.el8 ceph 24 k 2022-11-27T16:44:38.577 INFO:teuthology.orchestra.run.smithi145.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2022-11-27T16:44:38.577 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2022-11-27T16:44:38.578 INFO:teuthology.orchestra.run.smithi145.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2022-11-27T16:44:38.578 INFO:teuthology.orchestra.run.smithi145.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2022-11-27T16:44:38.578 INFO:teuthology.orchestra.run.smithi145.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2022-11-27T16:44:38.578 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite x86_64 2:18.0.0-594.g697b0d99.el8 ceph 137 k 2022-11-27T16:44:38.578 INFO:teuthology.orchestra.run.smithi145.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2022-11-27T16:44:38.579 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2022-11-27T16:44:38.579 INFO:teuthology.orchestra.run.smithi145.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2022-11-27T16:44:38.579 INFO:teuthology.orchestra.run.smithi145.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2022-11-27T16:44:38.579 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2022-11-27T16:44:38.580 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2022-11-27T16:44:38.580 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 474 k 2022-11-27T16:44:38.580 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2022-11-27T16:44:38.580 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2 x86_64 2:18.0.0-594.g697b0d99.el8 ceph 6.5 M 2022-11-27T16:44:38.580 INFO:teuthology.orchestra.run.smithi145.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2022-11-27T16:44:38.581 INFO:teuthology.orchestra.run.smithi145.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2022-11-27T16:44:38.581 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2022-11-27T16:44:38.581 INFO:teuthology.orchestra.run.smithi145.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2022-11-27T16:44:38.581 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2022-11-27T16:44:38.582 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2022-11-27T16:44:38.582 INFO:teuthology.orchestra.run.smithi145.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2022-11-27T16:44:38.582 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2022-11-27T16:44:38.582 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2022-11-27T16:44:38.582 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2022-11-27T16:44:38.583 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2022-11-27T16:44:38.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2022-11-27T16:44:38.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2022-11-27T16:44:38.583 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2022-11-27T16:44:38.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2022-11-27T16:44:38.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2022-11-27T16:44:38.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse x86_64 2:18.0.0-594.g697b0d99.el8 ceph 45 k 2022-11-27T16:44:38.584 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common x86_64 2:18.0.0-594.g697b0d99.el8 ceph 109 k 2022-11-27T16:44:38.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2022-11-27T16:44:38.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2022-11-27T16:44:38.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2022-11-27T16:44:38.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2022-11-27T16:44:38.585 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2022-11-27T16:44:38.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2022-11-27T16:44:38.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2022-11-27T16:44:38.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2022-11-27T16:44:38.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2022-11-27T16:44:38.586 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2022-11-27T16:44:38.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2022-11-27T16:44:38.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:44:38.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2022-11-27T16:44:38.587 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2022-11-27T16:44:38.588 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2022-11-27T16:44:38.588 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2022-11-27T16:44:38.588 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2022-11-27T16:44:38.588 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2022-11-27T16:44:38.588 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2022-11-27T16:44:38.589 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2022-11-27T16:44:38.589 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2022-11-27T16:44:38.589 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2022-11-27T16:44:38.589 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2022-11-27T16:44:38.590 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2022-11-27T16:44:38.590 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2022-11-27T16:44:38.590 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2022-11-27T16:44:38.590 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2022-11-27T16:44:38.590 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2022-11-27T16:44:38.591 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2022-11-27T16:44:38.591 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2022-11-27T16:44:38.591 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2022-11-27T16:44:38.591 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2022-11-27T16:44:38.592 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2022-11-27T16:44:38.592 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2022-11-27T16:44:38.592 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2022-11-27T16:44:38.592 INFO:teuthology.orchestra.run.smithi145.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2022-11-27T16:44:38.592 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2022-11-27T16:44:38.593 INFO:teuthology.orchestra.run.smithi145.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2022-11-27T16:44:38.593 INFO:teuthology.orchestra.run.smithi145.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2022-11-27T16:44:38.593 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2022-11-27T16:44:38.593 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2022-11-27T16:44:38.594 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2022-11-27T16:44:38.594 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2022-11-27T16:44:38.594 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2022-11-27T16:44:38.594 INFO:teuthology.orchestra.run.smithi145.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2022-11-27T16:44:38.595 INFO:teuthology.orchestra.run.smithi145.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2022-11-27T16:44:38.595 INFO:teuthology.orchestra.run.smithi145.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2022-11-27T16:44:38.595 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2022-11-27T16:44:38.595 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2022-11-27T16:44:38.595 INFO:teuthology.orchestra.run.smithi145.stdout:Enabling module streams: 2022-11-27T16:44:38.596 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql 10 2022-11-27T16:44:38.596 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:44:38.596 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction Summary 2022-11-27T16:44:38.596 INFO:teuthology.orchestra.run.smithi145.stdout:=========================================================================================================================== 2022-11-27T16:44:38.597 INFO:teuthology.orchestra.run.smithi145.stdout:Install 128 Packages 2022-11-27T16:44:38.597 INFO:teuthology.orchestra.run.smithi145.stdout:Upgrade 5 Packages 2022-11-27T16:44:38.597 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:44:38.597 INFO:teuthology.orchestra.run.smithi145.stdout:Total download size: 219 M 2022-11-27T16:44:38.597 INFO:teuthology.orchestra.run.smithi145.stdout:Downloading Packages: 2022-11-27T16:44:38.666 INFO:teuthology.orchestra.run.smithi201.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 52 kB/s | 6.4 kB 00:00 2022-11-27T16:44:38.696 INFO:teuthology.orchestra.run.smithi145.stdout:(1/133): ceph-18.0.0-594.g697b0d99.el8.x86_64.r 49 kB/s | 6.4 kB 00:00 2022-11-27T16:44:38.983 INFO:teuthology.orchestra.run.smithi201.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 2.6 MB/s | 845 kB 00:00 2022-11-27T16:44:39.058 INFO:teuthology.orchestra.run.smithi201.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 2.2 MB/s | 165 kB 00:00 2022-11-27T16:44:39.063 INFO:teuthology.orchestra.run.smithi145.stdout:(2/133): ceph-fuse-18.0.0-594.g697b0d99.el8.x86 2.2 MB/s | 845 kB 00:00 2022-11-27T16:44:39.189 INFO:teuthology.orchestra.run.smithi145.stdout:(3/133): ceph-immutable-object-cache-18.0.0-594 1.3 MB/s | 165 kB 00:00 2022-11-27T16:44:39.680 INFO:teuthology.orchestra.run.smithi145.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 4.7 MB/s | 5.2 MB 00:01 2022-11-27T16:44:39.692 INFO:teuthology.orchestra.run.smithi201.stdout:(4/133): ceph-base-18.0.0-594.g697b0d99.el8.x86 4.5 MB/s | 5.2 MB 00:01 2022-11-27T16:44:39.750 INFO:teuthology.orchestra.run.smithi201.stdout:(5/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 3.3 MB/s | 2.3 MB 00:00 2022-11-27T16:44:40.014 INFO:teuthology.orchestra.run.smithi145.stdout:(5/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 4.6 MB/s | 1.5 MB 00:00 2022-11-27T16:44:40.017 INFO:teuthology.orchestra.run.smithi201.stdout:(6/133): ceph-mgr-18.0.0-594.g697b0d99.el8.x86_ 4.7 MB/s | 1.5 MB 00:00 2022-11-27T16:44:40.056 INFO:teuthology.orchestra.run.smithi145.stdout:(6/133): ceph-mds-18.0.0-594.g697b0d99.el8.x86_ 2.6 MB/s | 2.3 MB 00:00 2022-11-27T16:44:40.793 INFO:teuthology.orchestra.run.smithi201.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 3.7 MB/s | 3.9 MB 00:01 2022-11-27T16:44:41.306 INFO:teuthology.orchestra.run.smithi145.stdout:(7/133): ceph-mon-18.0.0-594.g697b0d99.el8.x86_ 3.0 MB/s | 3.9 MB 00:01 2022-11-27T16:44:42.309 INFO:teuthology.orchestra.run.smithi201.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 6.5 MB/s | 24 MB 00:03 2022-11-27T16:44:42.368 INFO:teuthology.orchestra.run.smithi201.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 418 kB/s | 24 kB 00:00 2022-11-27T16:44:42.506 INFO:teuthology.orchestra.run.smithi145.stdout:(8/133): ceph-common-18.0.0-594.g697b0d99.el8.x 6.2 MB/s | 24 MB 00:03 2022-11-27T16:44:42.590 INFO:teuthology.orchestra.run.smithi145.stdout:(9/133): ceph-selinux-18.0.0-594.g697b0d99.el8. 292 kB/s | 24 kB 00:00 2022-11-27T16:44:43.968 INFO:teuthology.orchestra.run.smithi201.stdout:(10/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 4.3 MB/s | 17 MB 00:03 2022-11-27T16:44:44.068 INFO:teuthology.orchestra.run.smithi201.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 2.5 MB/s | 257 kB 00:00 2022-11-27T16:44:44.152 INFO:teuthology.orchestra.run.smithi201.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 364 kB/s | 30 kB 00:00 2022-11-27T16:44:44.377 INFO:teuthology.orchestra.run.smithi201.stdout:(13/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 3.3 MB/s | 767 kB 00:00 2022-11-27T16:44:44.461 INFO:teuthology.orchestra.run.smithi201.stdout:(14/133): libcephsqlite-18.0.0-594.g697b0d99.el 1.6 MB/s | 137 kB 00:00 2022-11-27T16:44:44.553 INFO:teuthology.orchestra.run.smithi201.stdout:(15/133): librados-devel-18.0.0-594.g697b0d99.e 1.5 MB/s | 139 kB 00:00 2022-11-27T16:44:44.720 INFO:teuthology.orchestra.run.smithi201.stdout:(16/133): libradosstriper1-18.0.0-594.g697b0d99 2.8 MB/s | 474 kB 00:00 2022-11-27T16:44:45.065 INFO:teuthology.orchestra.run.smithi145.stdout:(10/133): ceph-osd-18.0.0-594.g697b0d99.el8.x86 3.4 MB/s | 17 MB 00:05 2022-11-27T16:44:45.223 INFO:teuthology.orchestra.run.smithi145.stdout:(11/133): cephfs-mirror-18.0.0-594.g697b0d99.el 1.6 MB/s | 257 kB 00:00 2022-11-27T16:44:45.282 INFO:teuthology.orchestra.run.smithi145.stdout:(12/133): libcephfs-devel-18.0.0-594.g697b0d99. 522 kB/s | 30 kB 00:00 2022-11-27T16:44:45.362 INFO:teuthology.orchestra.run.smithi201.stdout:(17/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 4.0 MB/s | 18 MB 00:04 2022-11-27T16:44:45.429 INFO:teuthology.orchestra.run.smithi201.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 670 kB/s | 45 kB 00:00 2022-11-27T16:44:45.482 INFO:teuthology.orchestra.run.smithi145.stdout:(13/133): ceph-radosgw-18.0.0-594.g697b0d99.el8 4.4 MB/s | 18 MB 00:04 2022-11-27T16:44:45.504 INFO:teuthology.orchestra.run.smithi201.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.4 MB/s | 109 kB 00:00 2022-11-27T16:44:45.557 INFO:teuthology.orchestra.run.smithi145.stdout:(14/133): libcephfs2-18.0.0-594.g697b0d99.el8.x 2.7 MB/s | 767 kB 00:00 2022-11-27T16:44:45.583 INFO:teuthology.orchestra.run.smithi145.stdout:(15/133): libcephsqlite-18.0.0-594.g697b0d99.el 1.3 MB/s | 137 kB 00:00 2022-11-27T16:44:45.604 INFO:teuthology.orchestra.run.smithi201.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 1.9 MB/s | 190 kB 00:00 2022-11-27T16:44:45.642 INFO:teuthology.orchestra.run.smithi145.stdout:(16/133): librados-devel-18.0.0-594.g697b0d99.e 1.6 MB/s | 139 kB 00:00 2022-11-27T16:44:45.725 INFO:teuthology.orchestra.run.smithi145.stdout:(17/133): libradosstriper1-18.0.0-594.g697b0d99 3.3 MB/s | 474 kB 00:00 2022-11-27T16:44:45.763 INFO:teuthology.orchestra.run.smithi201.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 2.3 MB/s | 371 kB 00:00 2022-11-27T16:44:45.809 INFO:teuthology.orchestra.run.smithi145.stdout:(18/133): python3-ceph-argparse-18.0.0-594.g697 536 kB/s | 45 kB 00:00 2022-11-27T16:44:45.901 INFO:teuthology.orchestra.run.smithi145.stdout:(19/133): python3-ceph-common-18.0.0-594.g697b0 1.2 MB/s | 109 kB 00:00 2022-11-27T16:44:45.913 INFO:teuthology.orchestra.run.smithi201.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 2.3 MB/s | 357 kB 00:00 2022-11-27T16:44:45.976 INFO:teuthology.orchestra.run.smithi145.stdout:(20/133): python3-cephfs-18.0.0-594.g697b0d99.e 2.5 MB/s | 190 kB 00:00 2022-11-27T16:44:46.013 INFO:teuthology.orchestra.run.smithi201.stdout:(23/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 5.0 MB/s | 6.5 MB 00:01 2022-11-27T16:44:46.038 INFO:teuthology.orchestra.run.smithi201.stdout:(24/133): python3-rgw-18.0.0-594.g697b0d99.el8. 851 kB/s | 107 kB 00:00 2022-11-27T16:44:46.097 INFO:teuthology.orchestra.run.smithi201.stdout:(25/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.1 MB/s | 94 kB 00:00 2022-11-27T16:44:46.101 INFO:teuthology.orchestra.run.smithi145.stdout:(21/133): python3-rados-18.0.0-594.g697b0d99.el 2.9 MB/s | 371 kB 00:00 2022-11-27T16:44:46.164 INFO:teuthology.orchestra.run.smithi201.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 2.6 MB/s | 175 kB 00:00 2022-11-27T16:44:46.243 INFO:teuthology.orchestra.run.smithi145.stdout:(22/133): python3-rbd-18.0.0-594.g697b0d99.el8. 2.5 MB/s | 357 kB 00:00 2022-11-27T16:44:46.247 INFO:teuthology.orchestra.run.smithi201.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 274 kB/s | 23 kB 00:00 2022-11-27T16:44:46.302 INFO:teuthology.orchestra.run.smithi145.stdout:(23/133): python3-rgw-18.0.0-594.g697b0d99.el8. 1.8 MB/s | 107 kB 00:00 2022-11-27T16:44:46.339 INFO:teuthology.orchestra.run.smithi201.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.3 MB/s | 118 kB 00:00 2022-11-27T16:44:46.386 INFO:teuthology.orchestra.run.smithi145.stdout:(24/133): rbd-fuse-18.0.0-594.g697b0d99.el8.x86 1.1 MB/s | 94 kB 00:00 2022-11-27T16:44:46.798 INFO:teuthology.orchestra.run.smithi201.stdout:(29/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 3.6 MB/s | 1.7 MB 00:00 2022-11-27T16:44:46.923 INFO:teuthology.orchestra.run.smithi201.stdout:(30/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 3.9 MB/s | 3.5 MB 00:00 2022-11-27T16:44:47.024 INFO:teuthology.orchestra.run.smithi201.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.3 MB/s | 234 kB 00:00 2022-11-27T16:44:47.136 INFO:teuthology.orchestra.run.smithi201.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 417 kB/s | 47 kB 00:00 2022-11-27T16:44:47.194 INFO:teuthology.orchestra.run.smithi145.stdout:(25/133): rbd-mirror-18.0.0-594.g697b0d99.el8.x 4.3 MB/s | 3.5 MB 00:00 2022-11-27T16:44:47.216 INFO:teuthology.orchestra.run.smithi201.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 182 kB/s | 14 kB 00:00 2022-11-27T16:44:47.278 INFO:teuthology.orchestra.run.smithi145.stdout:(26/133): rbd-nbd-18.0.0-594.g697b0d99.el8.x86_ 2.1 MB/s | 175 kB 00:00 2022-11-27T16:44:47.308 INFO:teuthology.orchestra.run.smithi201.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 2.8 MB/s | 263 kB 00:00 2022-11-27T16:44:47.361 INFO:teuthology.orchestra.run.smithi145.stdout:(27/133): ceph-grafana-dashboards-18.0.0-594.g6 274 kB/s | 23 kB 00:00 2022-11-27T16:44:47.391 INFO:teuthology.orchestra.run.smithi201.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 975 kB/s | 81 kB 00:00 2022-11-27T16:44:47.437 INFO:teuthology.orchestra.run.smithi145.stdout:(28/133): ceph-mgr-cephadm-18.0.0-594.g697b0d99 1.5 MB/s | 118 kB 00:00 2022-11-27T16:44:47.475 INFO:teuthology.orchestra.run.smithi201.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 185 kB/s | 15 kB 00:00 2022-11-27T16:44:47.587 INFO:teuthology.orchestra.run.smithi145.stdout:(29/133): librgw2-18.0.0-594.g697b0d99.el8.x86_ 3.3 MB/s | 6.5 MB 00:01 2022-11-27T16:44:47.658 INFO:teuthology.orchestra.run.smithi201.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2022-11-27T16:44:47.709 INFO:teuthology.orchestra.run.smithi201.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2022-11-27T16:44:47.759 INFO:teuthology.orchestra.run.smithi201.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2022-11-27T16:44:47.846 INFO:teuthology.orchestra.run.smithi145.stdout:(30/133): ceph-mgr-dashboard-18.0.0-594.g697b0d 4.0 MB/s | 1.7 MB 00:00 2022-11-27T16:44:47.867 INFO:teuthology.orchestra.run.smithi201.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2022-11-27T16:44:47.910 INFO:teuthology.orchestra.run.smithi201.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2022-11-27T16:44:47.929 INFO:teuthology.orchestra.run.smithi145.stdout:(31/133): ceph-mgr-modules-core-18.0.0-594.g697 2.7 MB/s | 234 kB 00:00 2022-11-27T16:44:48.002 INFO:teuthology.orchestra.run.smithi201.stdout:(42/133): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2022-11-27T16:44:48.030 INFO:teuthology.orchestra.run.smithi145.stdout:(32/133): ceph-mgr-rook-18.0.0-594.g697b0d99.el 469 kB/s | 47 kB 00:00 2022-11-27T16:44:48.044 INFO:teuthology.orchestra.run.smithi201.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 791 kB/s | 33 kB 00:00 2022-11-27T16:44:48.086 INFO:teuthology.orchestra.run.smithi201.stdout:(44/133): python3-certifi-2018.10.15-7.el8.noar 391 kB/s | 16 kB 00:00 2022-11-27T16:44:48.113 INFO:teuthology.orchestra.run.smithi145.stdout:(33/133): ceph-prometheus-alerts-18.0.0-594.g69 173 kB/s | 14 kB 00:00 2022-11-27T16:44:48.136 INFO:teuthology.orchestra.run.smithi201.stdout:(45/133): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2022-11-27T16:44:48.197 INFO:teuthology.orchestra.run.smithi145.stdout:(34/133): ceph-volume-18.0.0-594.g697b0d99.el8. 3.1 MB/s | 263 kB 00:00 2022-11-27T16:44:48.219 INFO:teuthology.orchestra.run.smithi201.stdout:(46/133): python3-cherrypy-18.4.0-1.el8.noarch. 4.5 MB/s | 384 kB 00:00 2022-11-27T16:44:48.272 INFO:teuthology.orchestra.run.smithi145.stdout:(35/133): cephadm-18.0.0-594.g697b0d99.el8.noar 1.1 MB/s | 81 kB 00:00 2022-11-27T16:44:48.312 INFO:teuthology.orchestra.run.smithi201.stdout:(47/133): ceph-mgr-diskprediction-local-18.0.0- 4.9 MB/s | 7.4 MB 00:01 2022-11-27T16:44:48.322 INFO:teuthology.orchestra.run.smithi145.stdout:(36/133): cephfs-top-18.0.0-594.g697b0d99.el8.n 310 kB/s | 15 kB 00:00 2022-11-27T16:44:48.337 INFO:teuthology.orchestra.run.smithi201.stdout:(48/133): python3-google-auth-1.1.1-10.el8.noar 694 kB/s | 81 kB 00:00 2022-11-27T16:44:48.387 INFO:teuthology.orchestra.run.smithi201.stdout:(49/133): python3-jaraco-functools-2.0-4.el8.no 352 kB/s | 18 kB 00:00 2022-11-27T16:44:48.412 INFO:teuthology.orchestra.run.smithi201.stdout:(50/133): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2022-11-27T16:44:48.488 INFO:teuthology.orchestra.run.smithi201.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 648 kB/s | 49 kB 00:00 2022-11-27T16:44:48.506 INFO:teuthology.orchestra.run.smithi145.stdout:(37/133): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2022-11-27T16:44:48.556 INFO:teuthology.orchestra.run.smithi145.stdout:(38/133): liboath-2.6.2-3.el8.x86_64.rpm 1.1 MB/s | 59 kB 00:00 2022-11-27T16:44:48.606 INFO:teuthology.orchestra.run.smithi145.stdout:(39/133): libunwind-1.3.1-3.el8.x86_64.rpm 1.5 MB/s | 75 kB 00:00 2022-11-27T16:44:48.715 INFO:teuthology.orchestra.run.smithi145.stdout:(40/133): python3-asyncssh-2.7.0-2.el8.noarch.r 4.3 MB/s | 476 kB 00:00 2022-11-27T16:44:48.765 INFO:teuthology.orchestra.run.smithi145.stdout:(41/133): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 875 kB/s | 44 kB 00:00 2022-11-27T16:44:48.816 INFO:teuthology.orchestra.run.smithi145.stdout:(42/133): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2022-11-27T16:44:48.866 INFO:teuthology.orchestra.run.smithi145.stdout:(43/133): python3-cachetools-3.1.1-4.el8.noarch 658 kB/s | 33 kB 00:00 2022-11-27T16:44:48.908 INFO:teuthology.orchestra.run.smithi145.stdout:(44/133): python3-certifi-2018.10.15-7.el8.noar 392 kB/s | 16 kB 00:00 2022-11-27T16:44:48.975 INFO:teuthology.orchestra.run.smithi145.stdout:(45/133): python3-cheroot-8.5.2-1.el8.noarch.rp 2.5 MB/s | 173 kB 00:00 2022-11-27T16:44:49.005 INFO:teuthology.orchestra.run.smithi201.stdout:(52/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 8.3 MB/s | 55 MB 00:06 2022-11-27T16:44:49.030 INFO:teuthology.orchestra.run.smithi201.stdout:(53/133): python3-more-itertools-7.2.0-3.el8.no 108 kB/s | 59 kB 00:00 2022-11-27T16:44:49.072 INFO:teuthology.orchestra.run.smithi201.stdout:(54/133): python3-kubernetes-11.0.0-6.el8.noarc 1.7 MB/s | 1.2 MB 00:00 2022-11-27T16:44:49.117 INFO:teuthology.orchestra.run.smithi145.stdout:(46/133): ceph-mgr-diskprediction-local-18.0.0- 4.8 MB/s | 7.4 MB 00:01 2022-11-27T16:44:49.122 INFO:teuthology.orchestra.run.smithi201.stdout:(55/133): python3-portend-2.6-1.el8.noarch.rpm 313 kB/s | 16 kB 00:00 2022-11-27T16:44:49.142 INFO:teuthology.orchestra.run.smithi145.stdout:(47/133): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2022-11-27T16:44:49.184 INFO:teuthology.orchestra.run.smithi145.stdout:(48/133): python3-jaraco-6.2-6.el8.noarch.rpm 266 kB/s | 11 kB 00:00 2022-11-27T16:44:49.235 INFO:teuthology.orchestra.run.smithi145.stdout:(49/133): python3-jaraco-functools-2.0-4.el8.no 353 kB/s | 18 kB 00:00 2022-11-27T16:44:49.260 INFO:teuthology.orchestra.run.smithi145.stdout:(50/133): python3-google-auth-1.1.1-10.el8.noar 571 kB/s | 81 kB 00:00 2022-11-27T16:44:49.310 INFO:teuthology.orchestra.run.smithi145.stdout:(51/133): python3-logutils-0.3.5-11.el8.noarch. 970 kB/s | 49 kB 00:00 2022-11-27T16:44:49.331 INFO:teuthology.orchestra.run.smithi201.stdout:(56/133): python3-repoze-lru-0.7-6.el8.noarch.r 161 kB/s | 34 kB 00:00 2022-11-27T16:44:49.360 INFO:teuthology.orchestra.run.smithi145.stdout:(52/133): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2022-11-27T16:44:49.411 INFO:teuthology.orchestra.run.smithi145.stdout:(53/133): python3-natsort-7.1.1-2.el8.noarch.rp 1.2 MB/s | 60 kB 00:00 2022-11-27T16:44:49.444 INFO:teuthology.orchestra.run.smithi145.stdout:(54/133): python3-kubernetes-11.0.0-6.el8.noarc 5.6 MB/s | 1.2 MB 00:00 2022-11-27T16:44:49.456 INFO:teuthology.orchestra.run.smithi201.stdout:(57/133): python3-pecan-1.3.2-9.el8.noarch.rpm 664 kB/s | 283 kB 00:00 2022-11-27T16:44:49.481 INFO:teuthology.orchestra.run.smithi201.stdout:(58/133): python3-natsort-7.1.1-2.el8.noarch.rp 126 kB/s | 60 kB 00:00 2022-11-27T16:44:49.506 INFO:teuthology.orchestra.run.smithi201.stdout:(59/133): python3-routes-2.4.1-12.el8.noarch.rp 1.1 MB/s | 196 kB 00:00 2022-11-27T16:44:49.532 INFO:teuthology.orchestra.run.smithi201.stdout:(60/133): python3-rsa-4.8-1.el8.noarch.rpm 818 kB/s | 61 kB 00:00 2022-11-27T16:44:49.557 INFO:teuthology.orchestra.run.smithi201.stdout:(61/133): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2022-11-27T16:44:49.582 INFO:teuthology.orchestra.run.smithi201.stdout:(62/133): python3-singledispatch-3.4.0.3-18.el8 316 kB/s | 24 kB 00:00 2022-11-27T16:44:49.608 INFO:teuthology.orchestra.run.smithi201.stdout:(63/133): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2022-11-27T16:44:49.633 INFO:teuthology.orchestra.run.smithi201.stdout:(64/133): python3-trustme-0.6.0-4.el8.noarch.rp 365 kB/s | 27 kB 00:00 2022-11-27T16:44:49.658 INFO:teuthology.orchestra.run.smithi201.stdout:(65/133): python3-waitress-1.2.1-2.el8.1.noarch 3.1 MB/s | 241 kB 00:00 2022-11-27T16:44:49.692 INFO:teuthology.orchestra.run.smithi201.stdout:(66/133): python3-webob-1.8.5-1.el8.1.noarch.rp 2.9 MB/s | 251 kB 00:00 2022-11-27T16:44:49.717 INFO:teuthology.orchestra.run.smithi201.stdout:(67/133): python3-websocket-client-0.56.0-5.el8 730 kB/s | 61 kB 00:00 2022-11-27T16:44:49.762 INFO:teuthology.orchestra.run.smithi201.stdout:(68/133): python3-webtest-2.0.33-1.el8.noarch.r 821 kB/s | 85 kB 00:00 2022-11-27T16:44:49.792 INFO:teuthology.orchestra.run.smithi201.stdout:(69/133): python3-zc-lockfile-2.0-2.el8.noarch. 229 kB/s | 23 kB 00:00 2022-11-27T16:44:49.818 INFO:teuthology.orchestra.run.smithi201.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2022-11-27T16:44:49.876 INFO:teuthology.orchestra.run.smithi201.stdout:(71/133): pcre2-devel-10.32-2.el8.x86_64.rpm 7.1 MB/s | 605 kB 00:00 2022-11-27T16:44:49.910 INFO:teuthology.orchestra.run.smithi201.stdout:(72/133): fuse-2.9.7-15.el8.x86_64.rpm 900 kB/s | 83 kB 00:00 2022-11-27T16:44:49.944 INFO:teuthology.orchestra.run.smithi201.stdout:(73/133): libsepol-devel-2.9-3.el8.x86_64.rpm 1.3 MB/s | 87 kB 00:00 2022-11-27T16:44:49.953 INFO:teuthology.orchestra.run.smithi145.stdout:(55/133): ceph-test-18.0.0-594.g697b0d99.el8.x8 7.4 MB/s | 55 MB 00:07 2022-11-27T16:44:49.969 INFO:teuthology.orchestra.run.smithi201.stdout:(74/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2022-11-27T16:44:49.978 INFO:teuthology.orchestra.run.smithi145.stdout:(56/133): python3-portend-2.6-1.el8.noarch.rpm 29 kB/s | 16 kB 00:00 2022-11-27T16:44:50.003 INFO:teuthology.orchestra.run.smithi145.stdout:(57/133): python3-pecan-1.3.2-9.el8.noarch.rpm 477 kB/s | 283 kB 00:00 2022-11-27T16:44:50.011 INFO:teuthology.orchestra.run.smithi201.stdout:(75/133): thrift-0.13.0-2.el8.x86_64.rpm 5.9 MB/s | 1.7 MB 00:00 2022-11-27T16:44:50.045 INFO:teuthology.orchestra.run.smithi201.stdout:(76/133): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2022-11-27T16:44:50.061 INFO:teuthology.orchestra.run.smithi201.stdout:(77/133): libverto-devel-0.3.0-5.el8.x86_64.rpm 197 kB/s | 18 kB 00:00 2022-11-27T16:44:50.079 INFO:teuthology.orchestra.run.smithi145.stdout:(58/133): python3-rsa-4.8-1.el8.noarch.rpm 822 kB/s | 61 kB 00:00 2022-11-27T16:44:50.095 INFO:teuthology.orchestra.run.smithi201.stdout:(78/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 572 kB/s | 47 kB 00:00 2022-11-27T16:44:50.104 INFO:teuthology.orchestra.run.smithi145.stdout:(59/133): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2022-11-27T16:44:50.120 INFO:teuthology.orchestra.run.smithi201.stdout:(79/133): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2022-11-27T16:44:50.129 INFO:teuthology.orchestra.run.smithi145.stdout:(60/133): python3-repoze-lru-0.7-6.el8.noarch.r 191 kB/s | 34 kB 00:00 2022-11-27T16:44:50.154 INFO:teuthology.orchestra.run.smithi201.stdout:(80/133): python3-cryptography-3.2.1-5.el8.x86_ 5.0 MB/s | 559 kB 00:00 2022-11-27T16:44:50.155 INFO:teuthology.orchestra.run.smithi145.stdout:(61/133): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2022-11-27T16:44:50.180 INFO:teuthology.orchestra.run.smithi145.stdout:(62/133): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2022-11-27T16:44:50.187 INFO:teuthology.orchestra.run.smithi201.stdout:(81/133): python3-pycparser-2.14-14.el8.noarch. 1.2 MB/s | 109 kB 00:00 2022-11-27T16:44:50.205 INFO:teuthology.orchestra.run.smithi145.stdout:(63/133): python3-tempora-1.14.1-5.el8.noarch.r 387 kB/s | 29 kB 00:00 2022-11-27T16:44:50.221 INFO:teuthology.orchestra.run.smithi201.stdout:(82/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.5 MB/s | 560 kB 00:00 2022-11-27T16:44:50.230 INFO:teuthology.orchestra.run.smithi145.stdout:(64/133): python3-trustme-0.6.0-4.el8.noarch.rp 364 kB/s | 27 kB 00:00 2022-11-27T16:44:50.246 INFO:teuthology.orchestra.run.smithi201.stdout:(83/133): libquadmath-8.5.0-10.1.el8_6.x86_64.r 1.9 MB/s | 171 kB 00:00 2022-11-27T16:44:50.264 INFO:teuthology.orchestra.run.smithi145.stdout:(65/133): python3-waitress-1.2.1-2.el8.1.noarch 2.8 MB/s | 241 kB 00:00 2022-11-27T16:44:50.272 INFO:teuthology.orchestra.run.smithi201.stdout:(84/133): python3-jwt-1.6.1-2.el8.noarch.rpm 515 kB/s | 43 kB 00:00 2022-11-27T16:44:50.289 INFO:teuthology.orchestra.run.smithi145.stdout:(66/133): python3-websocket-client-0.56.0-5.el8 1.0 MB/s | 61 kB 00:00 2022-11-27T16:44:50.297 INFO:teuthology.orchestra.run.smithi201.stdout:(85/133): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 766 kB/s | 58 kB 00:00 2022-11-27T16:44:50.314 INFO:teuthology.orchestra.run.smithi145.stdout:(67/133): python3-webob-1.8.5-1.el8.1.noarch.rp 2.2 MB/s | 251 kB 00:00 2022-11-27T16:44:50.330 INFO:teuthology.orchestra.run.smithi201.stdout:(86/133): elfutils-libelf-devel-0.186-1.el8.x86 734 kB/s | 60 kB 00:00 2022-11-27T16:44:50.340 INFO:teuthology.orchestra.run.smithi145.stdout:(68/133): python3-webtest-2.0.33-1.el8.noarch.r 1.1 MB/s | 85 kB 00:00 2022-11-27T16:44:50.356 INFO:teuthology.orchestra.run.smithi201.stdout:(87/133): libaio-devel-0.3.112-1.el8.x86_64.rpm 326 kB/s | 19 kB 00:00 2022-11-27T16:44:50.365 INFO:teuthology.orchestra.run.smithi145.stdout:(69/133): python3-zc-lockfile-2.0-2.el8.noarch. 304 kB/s | 23 kB 00:00 2022-11-27T16:44:50.381 INFO:teuthology.orchestra.run.smithi201.stdout:(88/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2022-11-27T16:44:50.390 INFO:teuthology.orchestra.run.smithi145.stdout:(70/133): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2022-11-27T16:44:50.414 INFO:teuthology.orchestra.run.smithi201.stdout:(89/133): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.1 MB/s | 98 kB 00:00 2022-11-27T16:44:50.440 INFO:teuthology.orchestra.run.smithi201.stdout:(90/133): python3-requests-oauthlib-1.0.0-1.el8 742 kB/s | 43 kB 00:00 2022-11-27T16:44:50.441 INFO:teuthology.orchestra.run.smithi145.stdout:(71/133): postgresql-contrib-10.21-2.module+el8 11 MB/s | 808 kB 00:00 2022-11-27T16:44:50.465 INFO:teuthology.orchestra.run.smithi201.stdout:(91/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2022-11-27T16:44:50.501 INFO:teuthology.orchestra.run.smithi145.stdout:(72/133): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2022-11-27T16:44:50.507 INFO:teuthology.orchestra.run.smithi201.stdout:(92/133): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 785 kB/s | 72 kB 00:00 2022-11-27T16:44:50.532 INFO:teuthology.orchestra.run.smithi201.stdout:(93/133): libcom_err-devel-1.45.6-4.el8.x86_64. 420 kB/s | 39 kB 00:00 2022-11-27T16:44:50.558 INFO:teuthology.orchestra.run.smithi145.stdout:(73/133): postgresql-10.21-2.module+el8.6.0+153 13 MB/s | 1.5 MB 00:00 2022-11-27T16:44:50.616 INFO:teuthology.orchestra.run.smithi201.stdout:(94/133): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2022-11-27T16:44:50.641 INFO:teuthology.orchestra.run.smithi201.stdout:(95/133): keyutils-libs-devel-1.5.10-9.el8.x86_ 359 kB/s | 48 kB 00:00 2022-11-27T16:44:50.642 INFO:teuthology.orchestra.run.smithi145.stdout:(74/133): python3-numpy-1.14.3-10.el8.x86_64.rp 27 MB/s | 3.7 MB 00:00 2022-11-27T16:44:50.675 INFO:teuthology.orchestra.run.smithi201.stdout:(96/133): libgfortran-8.5.0-10.1.el8_6.x86_64.r 4.4 MB/s | 644 kB 00:00 2022-11-27T16:44:50.692 INFO:teuthology.orchestra.run.smithi145.stdout:(75/133): thrift-0.13.0-2.el8.x86_64.rpm 4.6 MB/s | 1.7 MB 00:00 2022-11-27T16:44:50.700 INFO:teuthology.orchestra.run.smithi201.stdout:(97/133): NetworkManager-initscripts-updown-1.3 1.6 MB/s | 138 kB 00:00 2022-11-27T16:44:50.734 INFO:teuthology.orchestra.run.smithi145.stdout:(76/133): python3-werkzeug-0.12.2-4.el8.noarch. 4.9 MB/s | 457 kB 00:00 2022-11-27T16:44:50.742 INFO:teuthology.orchestra.run.smithi201.stdout:(98/133): postgresql-contrib-10.21-2.module+el8 7.9 MB/s | 808 kB 00:00 2022-11-27T16:44:50.776 INFO:teuthology.orchestra.run.smithi201.stdout:(99/133): bison-3.0.4-10.el8.x86_64.rpm 6.7 MB/s | 688 kB 00:00 2022-11-27T16:44:50.826 INFO:teuthology.orchestra.run.smithi201.stdout:(100/133): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2022-11-27T16:44:50.826 INFO:teuthology.orchestra.run.smithi145.stdout:(77/133): postgresql-server-10.21-2.module+el8. 19 MB/s | 5.1 MB 00:00 2022-11-27T16:44:50.851 INFO:teuthology.orchestra.run.smithi145.stdout:(78/133): python3-markupsafe-0.23-19.el8.x86_64 245 kB/s | 39 kB 00:00 2022-11-27T16:44:50.868 INFO:teuthology.orchestra.run.smithi201.stdout:(101/133): python3-werkzeug-0.12.2-4.el8.noarch 11 MB/s | 457 kB 00:00 2022-11-27T16:44:50.876 INFO:teuthology.orchestra.run.smithi145.stdout:(79/133): python3-prettytable-0.7.2-14.el8.noar 309 kB/s | 44 kB 00:00 2022-11-27T16:44:50.918 INFO:teuthology.orchestra.run.smithi145.stdout:(80/133): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2022-11-27T16:44:50.935 INFO:teuthology.orchestra.run.smithi201.stdout:(102/133): python3-numpy-1.14.3-10.el8.x86_64.r 20 MB/s | 3.7 MB 00:00 2022-11-27T16:44:50.943 INFO:teuthology.orchestra.run.smithi145.stdout:(81/133): python3-jinja2-2.10.1-3.el8.noarch.rp 7.9 MB/s | 538 kB 00:00 2022-11-27T16:44:50.960 INFO:teuthology.orchestra.run.smithi201.stdout:(103/133): python3-markupsafe-0.23-19.el8.x86_6 423 kB/s | 39 kB 00:00 2022-11-27T16:44:50.985 INFO:teuthology.orchestra.run.smithi145.stdout:(82/133): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2022-11-27T16:44:50.994 INFO:teuthology.orchestra.run.smithi201.stdout:(104/133): python3-prettytable-0.7.2-14.el8.noa 751 kB/s | 44 kB 00:00 2022-11-27T16:44:51.013 INFO:teuthology.orchestra.run.smithi145.stdout:(83/133): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2022-11-27T16:44:51.046 INFO:teuthology.orchestra.run.smithi145.stdout:(84/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.7 MB/s | 157 kB 00:00 2022-11-27T16:44:51.069 INFO:teuthology.orchestra.run.smithi201.stdout:(105/133): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2022-11-27T16:44:51.086 INFO:teuthology.orchestra.run.smithi145.stdout:(85/133): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.4 MB/s | 103 kB 00:00 2022-11-27T16:44:51.094 INFO:teuthology.orchestra.run.smithi201.stdout:(106/133): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2022-11-27T16:44:51.128 INFO:teuthology.orchestra.run.smithi201.stdout:(107/133): python3-jinja2-2.10.1-3.el8.noarch.r 3.9 MB/s | 538 kB 00:00 2022-11-27T16:44:51.153 INFO:teuthology.orchestra.run.smithi201.stdout:(108/133): python3-numpy-f2py-1.14.3-10.el8.x86 3.8 MB/s | 225 kB 00:00 2022-11-27T16:44:51.187 INFO:teuthology.orchestra.run.smithi145.stdout:(86/133): openblas-threads-0.3.15-3.el8.x86_64. 20 MB/s | 4.9 MB 00:00 2022-11-27T16:44:51.195 INFO:teuthology.orchestra.run.smithi201.stdout:(109/133): libtool-2.4.6-25.el8.x86_64.rpm 5.5 MB/s | 709 kB 00:00 2022-11-27T16:44:51.220 INFO:teuthology.orchestra.run.smithi145.stdout:(87/133): socat-1.7.4.1-1.el8.x86_64.rpm 2.4 MB/s | 323 kB 00:00 2022-11-27T16:44:51.229 INFO:teuthology.orchestra.run.smithi201.stdout:(110/133): python3-mako-1.0.6-13.el8.noarch.rpm 2.0 MB/s | 157 kB 00:00 2022-11-27T16:44:51.271 INFO:teuthology.orchestra.run.smithi201.stdout:(111/133): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2022-11-27T16:44:51.295 INFO:teuthology.orchestra.run.smithi145.stdout:(88/133): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2022-11-27T16:44:51.313 INFO:teuthology.orchestra.run.smithi201.stdout:(112/133): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2022-11-27T16:44:51.321 INFO:teuthology.orchestra.run.smithi145.stdout:(89/133): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2022-11-27T16:44:51.354 INFO:teuthology.orchestra.run.smithi145.stdout:(90/133): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2022-11-27T16:44:51.379 INFO:teuthology.orchestra.run.smithi201.stdout:(113/133): openblas-threads-0.3.15-3.el8.x86_64 19 MB/s | 4.9 MB 00:00 2022-11-27T16:44:51.405 INFO:teuthology.orchestra.run.smithi145.stdout:(91/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-11-27T16:44:51.480 INFO:teuthology.orchestra.run.smithi201.stdout:(114/133): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2022-11-27T16:44:51.481 INFO:teuthology.orchestra.run.smithi145.stdout:(92/133): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2022-11-27T16:44:51.497 INFO:teuthology.orchestra.run.smithi145.stdout:(93/133): python3-pytz-2017.2-9.el8.noarch.rpm 591 kB/s | 54 kB 00:00 2022-11-27T16:44:51.505 INFO:teuthology.orchestra.run.smithi201.stdout:(115/133): flex-2.6.1-9.el8.x86_64.rpm 1.6 MB/s | 320 kB 00:00 2022-11-27T16:44:51.539 INFO:teuthology.orchestra.run.smithi145.stdout:(94/133): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.1 MB/s | 126 kB 00:00 2022-11-27T16:44:51.547 INFO:teuthology.orchestra.run.smithi201.stdout:(116/133): jq-1.6-3.el8.x86_64.rpm 3.0 MB/s | 202 kB 00:00 2022-11-27T16:44:51.555 INFO:teuthology.orchestra.run.smithi145.stdout:(95/133): python3-jsonpointer-1.10-11.el8.noarc 343 kB/s | 20 kB 00:00 2022-11-27T16:44:51.597 INFO:teuthology.orchestra.run.smithi201.stdout:(117/133): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2022-11-27T16:44:51.598 INFO:teuthology.orchestra.run.smithi145.stdout:(96/133): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2022-11-27T16:44:51.623 INFO:teuthology.orchestra.run.smithi145.stdout:(97/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.8 MB/s | 259 kB 00:00 2022-11-27T16:44:51.664 INFO:teuthology.orchestra.run.smithi201.stdout:(118/133): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2022-11-27T16:44:51.665 INFO:teuthology.orchestra.run.smithi145.stdout:(98/133): oniguruma-6.8.2-2.el8.x86_64.rpm 2.7 MB/s | 187 kB 00:00 2022-11-27T16:44:51.690 INFO:teuthology.orchestra.run.smithi145.stdout:(99/133): python3-jsonpatch-1.21-2.el8.noarch.r 402 kB/s | 27 kB 00:00 2022-11-27T16:44:51.698 INFO:teuthology.orchestra.run.smithi201.stdout:(119/133): python3-pytz-2017.2-9.el8.noarch.rpm 542 kB/s | 54 kB 00:00 2022-11-27T16:44:51.723 INFO:teuthology.orchestra.run.smithi201.stdout:(120/133): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.2 MB/s | 126 kB 00:00 2022-11-27T16:44:51.732 INFO:teuthology.orchestra.run.smithi145.stdout:(100/133): uuid-1.6.2-43.el8.x86_64.rpm 957 kB/s | 64 kB 00:00 2022-11-27T16:44:51.757 INFO:teuthology.orchestra.run.smithi201.stdout:(121/133): python3-jsonpointer-1.10-11.el8.noar 343 kB/s | 20 kB 00:00 2022-11-27T16:44:51.757 INFO:teuthology.orchestra.run.smithi145.stdout:(101/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.6 MB/s | 110 kB 00:00 2022-11-27T16:44:51.782 INFO:teuthology.orchestra.run.smithi201.stdout:(122/133): libpq-13.5-1.el8.x86_64.rpm 3.4 MB/s | 198 kB 00:00 2022-11-27T16:44:51.816 INFO:teuthology.orchestra.run.smithi201.stdout:(123/133): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2022-11-27T16:44:51.841 INFO:teuthology.orchestra.run.smithi201.stdout:(124/133): oniguruma-6.8.2-2.el8.x86_64.rpm 3.2 MB/s | 187 kB 00:00 2022-11-27T16:44:51.874 INFO:teuthology.orchestra.run.smithi201.stdout:(125/133): python3-jsonpatch-1.21-2.el8.noarch. 453 kB/s | 27 kB 00:00 2022-11-27T16:44:51.900 INFO:teuthology.orchestra.run.smithi201.stdout:(126/133): uuid-1.6.2-43.el8.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2022-11-27T16:44:51.925 INFO:teuthology.orchestra.run.smithi145.stdout:(102/133): python3-scipy-1.0.0-21.module+el8.5. 23 MB/s | 14 MB 00:00 2022-11-27T16:44:51.942 INFO:teuthology.orchestra.run.smithi201.stdout:(127/133): python3-pyasn1-modules-0.3.7-6.el8.n 1.6 MB/s | 110 kB 00:00 2022-11-27T16:44:52.025 INFO:teuthology.orchestra.run.smithi145.stdout:(103/133): fuse-2.9.7-15.el8.x86_64.rpm 311 kB/s | 83 kB 00:00 2022-11-27T16:44:52.084 INFO:teuthology.orchestra.run.smithi201.stdout:(128/133): python3-scipy-1.0.0-21.module+el8.5. 24 MB/s | 14 MB 00:00 2022-11-27T16:44:52.126 INFO:teuthology.orchestra.run.smithi201.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-11-27T16:44:52.184 INFO:teuthology.orchestra.run.smithi145.stdout:(104/133): pcre2-devel-10.32-2.el8.x86_64.rpm 1.3 MB/s | 605 kB 00:00 2022-11-27T16:44:52.187 INFO:teuthology.orchestra.run.smithi201.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-11-27T16:44:52.209 INFO:teuthology.orchestra.run.smithi145.stdout:(105/133): libsepol-devel-2.9-3.el8.x86_64.rpm 307 kB/s | 87 kB 00:00 2022-11-27T16:44:52.234 INFO:teuthology.orchestra.run.smithi145.stdout:(106/133): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.0 MB/s | 220 kB 00:00 2022-11-27T16:44:52.255 INFO:teuthology.orchestra.run.smithi201.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 11 MB/s | 798 kB 00:00 2022-11-27T16:44:52.276 INFO:teuthology.orchestra.run.smithi145.stdout:(107/133): libselinux-devel-2.9-5.el8.x86_64.rp 2.1 MB/s | 200 kB 00:00 2022-11-27T16:44:52.301 INFO:teuthology.orchestra.run.smithi145.stdout:(108/133): libverto-devel-0.3.0-5.el8.x86_64.rp 194 kB/s | 18 kB 00:00 2022-11-27T16:44:52.327 INFO:teuthology.orchestra.run.smithi145.stdout:(109/133): librabbitmq-0.9.0-3.el8.x86_64.rpm 520 kB/s | 47 kB 00:00 2022-11-27T16:44:52.368 INFO:teuthology.orchestra.run.smithi145.stdout:(110/133): python3-cryptography-3.2.1-5.el8.x86 5.9 MB/s | 559 kB 00:00 2022-11-27T16:44:52.394 INFO:teuthology.orchestra.run.smithi145.stdout:(111/133): python3-pycparser-2.14-14.el8.noarch 1.6 MB/s | 109 kB 00:00 2022-11-27T16:44:52.419 INFO:teuthology.orchestra.run.smithi145.stdout:(112/133): python3-oauthlib-2.1.0-1.el8.noarch. 1.3 MB/s | 155 kB 00:00 2022-11-27T16:44:52.461 INFO:teuthology.orchestra.run.smithi145.stdout:(113/133): krb5-devel-1.18.2-14.el8.x86_64.rpm 5.9 MB/s | 560 kB 00:00 2022-11-27T16:44:52.485 INFO:teuthology.orchestra.run.smithi201.stdout:(132/133): librbd1-18.0.0-594.g697b0d99.el8.x86 6.6 MB/s | 3.6 MB 00:00 2022-11-27T16:44:52.486 INFO:teuthology.orchestra.run.smithi145.stdout:(114/133): python3-jwt-1.6.1-2.el8.noarch.rpm 638 kB/s | 43 kB 00:00 2022-11-27T16:44:52.512 INFO:teuthology.orchestra.run.smithi145.stdout:(115/133): libquadmath-8.5.0-10.1.el8_6.x86_64. 1.4 MB/s | 171 kB 00:00 2022-11-27T16:44:52.545 INFO:teuthology.orchestra.run.smithi145.stdout:(116/133): zlib-devel-1.2.11-18.el8_5.x86_64.rp 688 kB/s | 58 kB 00:00 2022-11-27T16:44:52.551 INFO:teuthology.orchestra.run.smithi201.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 5.6 MB/s | 3.6 MB 00:00 2022-11-27T16:44:52.554 INFO:teuthology.orchestra.run.smithi201.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:44:52.554 INFO:teuthology.orchestra.run.smithi201.stdout:Total 16 MB/s | 219 MB 00:14 2022-11-27T16:44:52.571 INFO:teuthology.orchestra.run.smithi145.stdout:(117/133): elfutils-libelf-devel-0.186-1.el8.x8 716 kB/s | 60 kB 00:00 2022-11-27T16:44:52.621 INFO:teuthology.orchestra.run.smithi145.stdout:(118/133): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.2 MB/s | 238 kB 00:00 2022-11-27T16:44:52.654 INFO:teuthology.orchestra.run.smithi145.stdout:(119/133): libaio-devel-0.3.112-1.el8.x86_64.rp 172 kB/s | 19 kB 00:00 2022-11-27T16:44:52.679 INFO:teuthology.orchestra.run.smithi145.stdout:(120/133): libuuid-devel-2.32.1-35.el8.x86_64.r 903 kB/s | 98 kB 00:00 2022-11-27T16:44:52.705 INFO:teuthology.orchestra.run.smithi145.stdout:(121/133): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2022-11-27T16:44:52.738 INFO:teuthology.orchestra.run.smithi145.stdout:(122/133): python3-requests-oauthlib-1.0.0-1.el 510 kB/s | 43 kB 00:00 2022-11-27T16:44:52.764 INFO:teuthology.orchestra.run.smithi145.stdout:(123/133): libcom_err-devel-1.45.6-4.el8.x86_64 673 kB/s | 39 kB 00:00 2022-11-27T16:44:52.789 INFO:teuthology.orchestra.run.smithi145.stdout:(124/133): xfsprogs-devel-5.0.0-10.el8.x86_64.r 663 kB/s | 72 kB 00:00 2022-11-27T16:44:52.823 INFO:teuthology.orchestra.run.smithi145.stdout:(125/133): keyutils-libs-devel-1.5.10-9.el8.x86 824 kB/s | 48 kB 00:00 2022-11-27T16:44:52.856 INFO:teuthology.orchestra.run.smithi145.stdout:(126/133): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.6 MB/s | 644 kB 00:00 2022-11-27T16:44:52.882 INFO:teuthology.orchestra.run.smithi145.stdout:(127/133): NetworkManager-initscripts-updown-1. 2.3 MB/s | 138 kB 00:00 2022-11-27T16:44:52.932 INFO:teuthology.orchestra.run.smithi145.stdout:(128/133): openssl-devel-1.1.1k-7.el8_6.x86_64. 12 MB/s | 2.3 MB 00:00 2022-11-27T16:44:52.974 INFO:teuthology.orchestra.run.smithi145.stdout:(129/133): openssl-1.1.1k-7.el8_6.x86_64.rpm 17 MB/s | 709 kB 00:00 2022-11-27T16:44:53.036 INFO:teuthology.orchestra.run.smithi145.stdout:(130/133): openssl-libs-1.1.1k-7.el8_6.x86_64.r 25 MB/s | 1.5 MB 00:00 2022-11-27T16:44:53.083 INFO:teuthology.orchestra.run.smithi145.stdout:(131/133): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2022-11-27T16:44:53.391 INFO:teuthology.orchestra.run.smithi145.stdout:(132/133): librbd1-18.0.0-594.g697b0d99.el8.x86 7.0 MB/s | 3.6 MB 00:00 2022-11-27T16:44:53.408 INFO:teuthology.orchestra.run.smithi201.stdout:Running transaction check 2022-11-27T16:44:53.450 INFO:teuthology.orchestra.run.smithi145.stdout:(133/133): librados2-18.0.0-594.g697b0d99.el8.x 6.1 MB/s | 3.6 MB 00:00 2022-11-27T16:44:53.453 INFO:teuthology.orchestra.run.smithi145.stdout:-------------------------------------------------------------------------------- 2022-11-27T16:44:53.453 INFO:teuthology.orchestra.run.smithi145.stdout:Total 15 MB/s | 219 MB 00:14 2022-11-27T16:44:53.568 INFO:teuthology.orchestra.run.smithi201.stdout:Transaction check succeeded. 2022-11-27T16:44:53.569 INFO:teuthology.orchestra.run.smithi201.stdout:Running transaction test 2022-11-27T16:44:54.292 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction check 2022-11-27T16:44:54.450 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction check succeeded. 2022-11-27T16:44:54.451 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction test 2022-11-27T16:44:54.742 INFO:teuthology.orchestra.run.smithi201.stdout:Transaction test succeeded. 2022-11-27T16:44:54.747 INFO:teuthology.orchestra.run.smithi201.stdout:Running transaction 2022-11-27T16:44:55.611 INFO:teuthology.orchestra.run.smithi145.stdout:Transaction test succeeded. 2022-11-27T16:44:55.615 INFO:teuthology.orchestra.run.smithi145.stdout:Running transaction 2022-11-27T16:44:59.696 INFO:teuthology.orchestra.run.smithi201.stdout: Preparing : 1/1 2022-11-27T16:45:00.195 INFO:teuthology.orchestra.run.smithi201.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:45:00.412 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:45:00.602 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T16:45:00.742 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T16:45:00.910 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 4/138 2022-11-27T16:45:00.979 INFO:teuthology.orchestra.run.smithi145.stdout: Preparing : 1/1 2022-11-27T16:45:01.079 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T16:45:01.176 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 5/138 2022-11-27T16:45:01.215 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:45:01.338 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:45:01.429 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openssl-libs-1:1.1.1k-7.el8_6.x86_64 1/138 2022-11-27T16:45:01.604 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/138 2022-11-27T16:45:01.745 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/138 2022-11-27T16:45:01.758 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:45:01.898 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T16:45:01.968 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/138 2022-11-27T16:45:01.979 INFO:teuthology.orchestra.run.smithi201.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:45:02.156 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 5/138 2022-11-27T16:45:02.298 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:45:02.350 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:45:02.554 INFO:teuthology.orchestra.run.smithi201.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:45:02.678 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:45:02.689 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 6/138 2022-11-27T16:45:02.879 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:45:02.914 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:45:02.975 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:45:03.185 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:45:03.288 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:45:03.415 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T16:45:03.456 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:45:03.582 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T16:45:03.583 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:45:03.773 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:45:03.872 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:45:03.954 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T16:45:04.053 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:45:04.174 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T16:45:04.260 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libpq-13.5-1.el8.x86_64 11/138 2022-11-27T16:45:04.337 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:45:04.422 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:45:04.451 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 12/138 2022-11-27T16:45:04.578 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:45:04.653 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:45:04.832 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 13/138 2022-11-27T16:45:04.856 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 17/138 2022-11-27T16:45:05.018 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 14/138 2022-11-27T16:45:05.078 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 18/138 2022-11-27T16:45:05.173 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:45:05.277 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:45:05.286 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 19/138 2022-11-27T16:45:05.439 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:45:05.458 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 20/138 2022-11-27T16:45:05.508 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 16/138 2022-11-27T16:45:05.653 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 17/138 2022-11-27T16:45:05.747 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T16:45:05.874 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 18/138 2022-11-27T16:45:05.886 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T16:45:06.014 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T16:45:06.031 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 21/138 2022-11-27T16:45:06.170 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T16:45:06.633 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 19/138 2022-11-27T16:45:06.787 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 22/138 2022-11-27T16:45:06.805 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T16:45:07.003 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T16:45:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 20/138 2022-11-27T16:45:07.564 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T16:45:07.602 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 23/138 2022-11-27T16:45:07.787 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T16:45:08.240 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 21/138 2022-11-27T16:45:08.439 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 24/138 2022-11-27T16:45:08.695 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 22/138 2022-11-27T16:45:08.749 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/138 2022-11-27T16:45:08.859 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 23/138 2022-11-27T16:45:09.043 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T16:45:09.127 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 24/138 2022-11-27T16:45:09.176 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T16:45:09.321 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T16:45:09.386 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/138 2022-11-27T16:45:09.564 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T16:45:09.661 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 26/138 2022-11-27T16:45:09.761 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T16:45:09.811 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 27/138 2022-11-27T16:45:09.932 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 28/138 2022-11-27T16:45:10.179 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 29/138 2022-11-27T16:45:11.093 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 30/138 2022-11-27T16:45:11.425 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/138 2022-11-27T16:45:11.476 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 30/138 2022-11-27T16:45:11.699 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/138 2022-11-27T16:45:11.828 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 31/138 2022-11-27T16:45:11.855 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T16:45:12.763 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 32/138 2022-11-27T16:45:13.089 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/138 2022-11-27T16:45:13.344 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/138 2022-11-27T16:45:13.546 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T16:45:13.570 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 33/138 2022-11-27T16:45:13.761 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T16:45:13.897 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 34/138 2022-11-27T16:45:13.991 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T16:45:14.140 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rsa-4.8-1.el8.noarch 35/138 2022-11-27T16:45:14.179 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T16:45:14.338 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 36/138 2022-11-27T16:45:14.386 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T16:45:14.568 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 37/138 2022-11-27T16:45:14.569 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T16:45:14.649 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/138 2022-11-27T16:45:14.766 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 38/138 2022-11-27T16:45:14.843 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T16:45:14.915 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 41/138 2022-11-27T16:45:14.958 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 39/138 2022-11-27T16:45:15.075 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 40/138 2022-11-27T16:45:15.083 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : jq-1.6-3.el8.x86_64 42/138 2022-11-27T16:45:15.234 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 43/138 2022-11-27T16:45:15.257 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 41/138 2022-11-27T16:45:15.436 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/138 2022-11-27T16:45:15.477 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 44/138 2022-11-27T16:45:15.635 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 43/138 2022-11-27T16:45:15.735 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : automake-1.16.1-7.el8.noarch 45/138 2022-11-27T16:45:15.842 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 44/138 2022-11-27T16:45:15.880 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 46/138 2022-11-27T16:45:16.022 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 47/138 2022-11-27T16:45:16.188 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 48/138 2022-11-27T16:45:16.223 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 45/138 2022-11-27T16:45:16.392 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 49/138 2022-11-27T16:45:16.523 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 46/138 2022-11-27T16:45:16.599 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 50/138 2022-11-27T16:45:16.730 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 47/138 2022-11-27T16:45:16.806 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/138 2022-11-27T16:45:16.981 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 48/138 2022-11-27T16:45:17.192 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/138 2022-11-27T16:45:17.221 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 49/138 2022-11-27T16:45:17.516 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 50/138 2022-11-27T16:45:17.517 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/138 2022-11-27T16:45:17.686 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 51/138 2022-11-27T16:45:17.704 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/138 2022-11-27T16:45:17.846 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 52/138 2022-11-27T16:45:17.920 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/138 2022-11-27T16:45:18.081 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 53/138 2022-11-27T16:45:18.188 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/138 2022-11-27T16:45:18.214 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 54/138 2022-11-27T16:45:18.516 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/138 2022-11-27T16:45:18.522 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/138 2022-11-27T16:45:18.723 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/138 2022-11-27T16:45:18.734 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/138 2022-11-27T16:45:18.927 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/138 2022-11-27T16:45:18.963 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 57/138 2022-11-27T16:45:19.153 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/138 2022-11-27T16:45:19.181 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 58/138 2022-11-27T16:45:19.312 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 61/138 2022-11-27T16:45:19.412 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 59/138 2022-11-27T16:45:19.587 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : fuse-2.9.7-15.el8.x86_64 60/138 2022-11-27T16:45:19.628 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 62/138 2022-11-27T16:45:19.753 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T16:45:19.837 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/138 2022-11-27T16:45:19.838 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 63/138 2022-11-27T16:45:20.018 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T16:45:20.102 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 64/138 2022-11-27T16:45:20.120 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 62/138 2022-11-27T16:45:20.302 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : jq-1.6-3.el8.x86_64 63/138 2022-11-27T16:45:20.354 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/138 2022-11-27T16:45:20.460 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 64/138 2022-11-27T16:45:20.618 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/138 2022-11-27T16:45:20.706 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 65/138 2022-11-27T16:45:20.801 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : fuse-2.9.7-15.el8.x86_64 67/138 2022-11-27T16:45:20.945 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : automake-1.16.1-7.el8.noarch 66/138 2022-11-27T16:45:20.991 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T16:45:21.088 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 67/138 2022-11-27T16:45:21.161 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:45:21.256 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 68/138 2022-11-27T16:45:21.408 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:45:21.454 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T16:45:21.703 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 70/138 2022-11-27T16:45:21.874 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T16:45:21.902 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 71/138 2022-11-27T16:45:22.049 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T16:45:22.055 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 72/138 2022-11-27T16:45:22.186 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T16:45:22.239 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 73/138 2022-11-27T16:45:22.403 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T16:45:22.408 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 74/138 2022-11-27T16:45:22.582 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T16:45:22.629 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 75/138 2022-11-27T16:45:22.747 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T16:45:22.836 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 76/138 2022-11-27T16:45:22.931 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T16:45:22.996 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 77/138 2022-11-27T16:45:23.058 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T16:45:23.157 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 78/138 2022-11-27T16:45:23.247 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T16:45:23.353 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 79/138 2022-11-27T16:45:23.423 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T16:45:23.554 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/138 2022-11-27T16:45:23.608 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T16:45:23.731 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 81/138 2022-11-27T16:45:23.821 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T16:45:23.994 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-portend-2.6-1.el8.noarch 82/138 2022-11-27T16:45:24.018 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T16:45:24.167 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T16:45:24.182 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 83/138 2022-11-27T16:45:24.381 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 84/138 2022-11-27T16:45:24.408 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T16:45:24.580 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 85/138 2022-11-27T16:45:25.448 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T16:45:25.653 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 86/138 2022-11-27T16:45:25.697 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T16:45:25.911 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 87/138 2022-11-27T16:45:25.919 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T16:45:26.126 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 88/138 2022-11-27T16:45:26.200 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T16:45:26.413 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 89/138 2022-11-27T16:45:26.494 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T16:45:26.668 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T16:45:26.717 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 90/138 2022-11-27T16:45:26.899 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 91/138 2022-11-27T16:45:27.422 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T16:45:27.568 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:45:27.633 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:45:27.727 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 92/138 2022-11-27T16:45:27.844 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T16:45:27.874 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:45:27.946 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 93/138 2022-11-27T16:45:28.158 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 94/138 2022-11-27T16:45:31.577 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:45:31.785 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:45:31.937 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:45:32.149 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:45:32.341 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:45:32.601 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:45:32.667 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:45:32.704 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 95/138 2022-11-27T16:45:32.793 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:45:32.973 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:45:33.064 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 96/138 2022-11-27T16:45:33.182 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:45:51.539 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:45:51.539 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /sys 2022-11-27T16:45:51.540 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /proc 2022-11-27T16:45:51.540 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /dev 2022-11-27T16:45:51.540 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /run 2022-11-27T16:45:51.541 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /mnt 2022-11-27T16:45:51.541 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /var/tmp 2022-11-27T16:45:51.541 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /home 2022-11-27T16:45:51.542 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /tmp 2022-11-27T16:45:51.542 INFO:teuthology.orchestra.run.smithi145.stdout:skipping the directory /dev 2022-11-27T16:45:51.543 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:45:51.890 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:45:51.974 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 97/138 2022-11-27T16:45:51.974 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /sys 2022-11-27T16:45:51.975 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /proc 2022-11-27T16:45:51.975 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /dev 2022-11-27T16:45:51.975 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /run 2022-11-27T16:45:51.975 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /mnt 2022-11-27T16:45:51.976 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /var/tmp 2022-11-27T16:45:51.976 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /home 2022-11-27T16:45:51.976 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /tmp 2022-11-27T16:45:51.976 INFO:teuthology.orchestra.run.smithi201.stdout:skipping the directory /dev 2022-11-27T16:45:51.977 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:45:52.329 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:45:53.374 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:45:53.564 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:45:53.779 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:45:53.808 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 98/138 2022-11-27T16:45:53.956 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:45:54.019 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:45:54.235 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 99/138 2022-11-27T16:45:54.336 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:45:54.446 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:45:54.489 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:45:54.506 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:45:54.828 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 100/138 2022-11-27T16:45:55.004 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:45:55.021 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 101/138 2022-11-27T16:45:56.028 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:45:56.157 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:45:56.261 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T16:45:56.408 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T16:45:56.580 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:45:56.788 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:45:56.806 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:45:56.963 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 102/138 2022-11-27T16:45:56.979 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:45:57.101 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 103/138 2022-11-27T16:45:57.273 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 104/138 2022-11-27T16:45:57.428 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:45:57.627 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 105/138 2022-11-27T16:45:57.852 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:45:57.939 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:45:58.104 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:45:58.192 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:45:58.531 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T16:45:58.685 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:45:58.720 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:45:58.810 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 106/138 2022-11-27T16:45:58.904 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:45:58.916 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:45:58.986 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:45:59.076 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 107/138 2022-11-27T16:45:59.528 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 108/138 2022-11-27T16:45:59.713 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:45:59.747 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 109/138 2022-11-27T16:45:59.911 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:45:59.922 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 110/138 2022-11-27T16:46:00.623 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T16:46:00.821 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:46:00.855 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:46:01.021 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:46:01.062 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:46:01.204 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:46:01.550 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:46:01.677 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 111/138 2022-11-27T16:46:01.712 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:46:01.894 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:46:01.928 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 112/138 2022-11-27T16:46:02.086 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:46:02.128 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-594.g697b0d 113/138 2022-11-27T16:46:02.294 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:46:02.639 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 114/138 2022-11-27T16:46:02.803 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:46:06.801 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:46:07.071 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T16:46:07.211 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T16:46:07.398 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 117/138 2022-11-27T16:46:07.692 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 118/138 2022-11-27T16:46:07.886 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 115/138 2022-11-27T16:46:08.208 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 116/138 2022-11-27T16:46:08.464 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 119/138 2022-11-27T16:46:08.739 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 120/138 2022-11-27T16:46:08.907 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 121/138 2022-11-27T16:46:09.027 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 117/138 2022-11-27T16:46:09.082 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T16:46:09.204 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T16:46:09.304 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/138 2022-11-27T16:46:09.528 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 119/138 2022-11-27T16:46:09.709 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T16:46:09.897 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 120/138 2022-11-27T16:46:10.164 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 121/138 2022-11-27T16:46:10.347 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 122/138 2022-11-27T16:46:10.477 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 123/138 2022-11-27T16:46:11.238 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:46:11.461 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:46:11.478 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:46:11.654 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T16:46:11.918 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T16:46:12.193 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T16:46:12.493 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:46:12.591 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T16:46:12.784 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:46:12.801 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 124/138 2022-11-27T16:46:12.887 INFO:teuthology.orchestra.run.smithi145.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T16:46:12.976 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 125/138 2022-11-27T16:46:13.052 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T16:46:13.061 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/138 2022-11-27T16:46:13.221 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/138 2022-11-27T16:46:13.233 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 126/138 2022-11-27T16:46:13.418 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T16:46:13.494 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 132/138 2022-11-27T16:46:13.525 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 127/138 2022-11-27T16:46:13.702 INFO:teuthology.orchestra.run.smithi145.stdout: Installing : bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T16:46:13.717 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 133/138 2022-11-27T16:46:13.881 INFO:teuthology.orchestra.run.smithi201.stdout: Upgrading : openssl-1:1.1.1k-7.el8_6.x86_64 128/138 2022-11-27T16:46:13.895 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:46:13.895 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:46:14.071 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:46:14.072 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:46:14.210 INFO:teuthology.orchestra.run.smithi201.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 129/138 2022-11-27T16:46:14.304 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:46:14.304 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:46:14.483 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T16:46:14.556 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:46:14.557 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/138 2022-11-27T16:46:14.767 INFO:teuthology.orchestra.run.smithi145.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:46:14.767 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T16:46:14.779 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/138 2022-11-27T16:46:14.950 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T16:46:14.958 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 132/138 2022-11-27T16:46:15.121 INFO:teuthology.orchestra.run.smithi201.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 133/138 2022-11-27T16:46:15.310 INFO:teuthology.orchestra.run.smithi201.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:46:15.311 INFO:teuthology.orchestra.run.smithi201.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:46:15.336 INFO:teuthology.orchestra.run.smithi145.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:46:15.336 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T16:46:15.337 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T16:46:15.337 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T16:46:15.337 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T16:46:15.337 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T16:46:15.338 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T16:46:15.338 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:46:15.338 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:46:15.338 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:46:15.338 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:46:15.339 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T16:46:15.339 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T16:46:15.339 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T16:46:15.339 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T16:46:15.340 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:46:15.340 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T16:46:15.340 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T16:46:15.340 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T16:46:15.341 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T16:46:15.341 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T16:46:15.341 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T16:46:15.341 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T16:46:15.341 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T16:46:15.342 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T16:46:15.342 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T16:46:15.342 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T16:46:15.342 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T16:46:15.343 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T16:46:15.343 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T16:46:15.343 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T16:46:15.343 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T16:46:15.343 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T16:46:15.344 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T16:46:15.344 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T16:46:15.344 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T16:46:15.345 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T16:46:15.345 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T16:46:15.345 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T16:46:15.345 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T16:46:15.345 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T16:46:15.346 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T16:46:15.346 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T16:46:15.346 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T16:46:15.346 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T16:46:15.347 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T16:46:15.347 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T16:46:15.347 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T16:46:15.347 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T16:46:15.348 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T16:46:15.348 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T16:46:15.348 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T16:46:15.348 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T16:46:15.349 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T16:46:15.349 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T16:46:15.349 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T16:46:15.349 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T16:46:15.350 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T16:46:15.350 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T16:46:15.350 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T16:46:15.351 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T16:46:15.351 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T16:46:15.351 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T16:46:15.351 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T16:46:15.352 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T16:46:15.352 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T16:46:15.352 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T16:46:15.352 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T16:46:15.353 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T16:46:15.353 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:46:15.353 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T16:46:15.353 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T16:46:15.354 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 72/138 2022-11-27T16:46:15.354 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : bison-3.0.4-10.el8.x86_64 73/138 2022-11-27T16:46:15.354 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 74/138 2022-11-27T16:46:15.354 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 75/138 2022-11-27T16:46:15.354 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 76/138 2022-11-27T16:46:15.355 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/138 2022-11-27T16:46:15.355 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 78/138 2022-11-27T16:46:15.356 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 79/138 2022-11-27T16:46:15.356 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 80/138 2022-11-27T16:46:15.356 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 81/138 2022-11-27T16:46:15.356 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 82/138 2022-11-27T16:46:15.356 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 83/138 2022-11-27T16:46:15.357 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 84/138 2022-11-27T16:46:15.357 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 85/138 2022-11-27T16:46:15.357 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 86/138 2022-11-27T16:46:15.357 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 87/138 2022-11-27T16:46:15.358 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 88/138 2022-11-27T16:46:15.358 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : flex-2.6.1-9.el8.x86_64 89/138 2022-11-27T16:46:15.358 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 90/138 2022-11-27T16:46:15.358 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : jq-1.6-3.el8.x86_64 91/138 2022-11-27T16:46:15.359 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 92/138 2022-11-27T16:46:15.359 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : automake-1.16.1-7.el8.noarch 93/138 2022-11-27T16:46:15.359 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 94/138 2022-11-27T16:46:15.359 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 95/138 2022-11-27T16:46:15.360 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 96/138 2022-11-27T16:46:15.360 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libpq-13.5-1.el8.x86_64 97/138 2022-11-27T16:46:15.360 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 98/138 2022-11-27T16:46:15.360 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 99/138 2022-11-27T16:46:15.360 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 100/138 2022-11-27T16:46:15.361 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 101/138 2022-11-27T16:46:15.361 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 102/138 2022-11-27T16:46:15.361 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 103/138 2022-11-27T16:46:15.361 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 104/138 2022-11-27T16:46:15.362 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 105/138 2022-11-27T16:46:15.362 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/138 2022-11-27T16:46:15.362 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 107/138 2022-11-27T16:46:15.362 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 108/138 2022-11-27T16:46:15.363 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 109/138 2022-11-27T16:46:15.363 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 110/138 2022-11-27T16:46:15.363 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 111/138 2022-11-27T16:46:15.363 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 112/138 2022-11-27T16:46:15.364 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 113/138 2022-11-27T16:46:15.364 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 114/138 2022-11-27T16:46:15.364 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 115/138 2022-11-27T16:46:15.365 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/138 2022-11-27T16:46:15.365 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 117/138 2022-11-27T16:46:15.365 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/138 2022-11-27T16:46:15.365 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 119/138 2022-11-27T16:46:15.366 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 120/138 2022-11-27T16:46:15.366 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 121/138 2022-11-27T16:46:15.366 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 122/138 2022-11-27T16:46:15.366 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 123/138 2022-11-27T16:46:15.367 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 124/138 2022-11-27T16:46:15.367 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 125/138 2022-11-27T16:46:15.367 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 126/138 2022-11-27T16:46:15.368 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 127/138 2022-11-27T16:46:15.368 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/138 2022-11-27T16:46:15.368 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T16:46:15.368 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T16:46:15.369 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T16:46:15.369 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T16:46:15.369 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T16:46:15.369 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:46:15.369 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T16:46:15.370 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T16:46:15.370 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T16:46:15.520 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 135/138 2022-11-27T16:46:15.520 INFO:teuthology.orchestra.run.smithi201.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:46:15.753 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 136/138 2022-11-27T16:46:15.753 INFO:teuthology.orchestra.run.smithi201.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:46:16.054 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/138 2022-11-27T16:46:16.240 INFO:teuthology.orchestra.run.smithi201.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:46:16.774 INFO:teuthology.orchestra.run.smithi201.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:46:16.774 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-2:18.0.0-594.g697b0d99.el8.x86_64 1/138 2022-11-27T16:46:16.775 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2/138 2022-11-27T16:46:16.775 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 3/138 2022-11-27T16:46:16.775 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 4/138 2022-11-27T16:46:16.776 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-594.g697b0d 5/138 2022-11-27T16:46:16.776 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 6/138 2022-11-27T16:46:16.776 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 7/138 2022-11-27T16:46:16.776 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 8/138 2022-11-27T16:46:16.776 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 9/138 2022-11-27T16:46:16.777 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 10/138 2022-11-27T16:46:16.777 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 11/138 2022-11-27T16:46:16.777 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 12/138 2022-11-27T16:46:16.778 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 13/138 2022-11-27T16:46:16.778 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_6 14/138 2022-11-27T16:46:16.778 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 15/138 2022-11-27T16:46:16.778 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 16/138 2022-11-27T16:46:16.778 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 17/138 2022-11-27T16:46:16.779 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_ 18/138 2022-11-27T16:46:16.779 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 19/138 2022-11-27T16:46:16.780 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8 20/138 2022-11-27T16:46:16.780 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x 21/138 2022-11-27T16:46:16.781 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 22/138 2022-11-27T16:46:16.781 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 23/138 2022-11-27T16:46:16.782 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 24/138 2022-11-27T16:46:16.782 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 25/138 2022-11-27T16:46:16.782 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 26/138 2022-11-27T16:46:16.783 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 27/138 2022-11-27T16:46:16.783 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 28/138 2022-11-27T16:46:16.784 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.e 29/138 2022-11-27T16:46:16.784 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noar 30/138 2022-11-27T16:46:16.785 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.no 31/138 2022-11-27T16:46:16.809 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-594.g697b 32/138 2022-11-27T16:46:16.810 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8 33/138 2022-11-27T16:46:16.810 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 34/138 2022-11-27T16:46:16.811 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el 35/138 2022-11-27T16:46:16.811 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 36/138 2022-11-27T16:46:16.811 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : cephadm-2:18.0.0-594.g697b0d99.el8.noarch 37/138 2022-11-27T16:46:16.812 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 38/138 2022-11-27T16:46:16.812 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/138 2022-11-27T16:46:16.813 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/138 2022-11-27T16:46:16.813 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/138 2022-11-27T16:46:16.813 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/138 2022-11-27T16:46:16.814 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/138 2022-11-27T16:46:16.814 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/138 2022-11-27T16:46:16.814 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/138 2022-11-27T16:46:16.815 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/138 2022-11-27T16:46:16.815 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/138 2022-11-27T16:46:16.816 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/138 2022-11-27T16:46:16.816 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/138 2022-11-27T16:46:16.817 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/138 2022-11-27T16:46:16.817 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/138 2022-11-27T16:46:16.818 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/138 2022-11-27T16:46:16.818 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/138 2022-11-27T16:46:16.819 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/138 2022-11-27T16:46:16.819 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/138 2022-11-27T16:46:16.820 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/138 2022-11-27T16:46:16.820 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/138 2022-11-27T16:46:16.820 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/138 2022-11-27T16:46:16.821 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/138 2022-11-27T16:46:16.821 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/138 2022-11-27T16:46:16.822 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/138 2022-11-27T16:46:16.822 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/138 2022-11-27T16:46:16.823 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/138 2022-11-27T16:46:16.823 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/138 2022-11-27T16:46:16.823 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 65/138 2022-11-27T16:46:16.824 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/138 2022-11-27T16:46:16.824 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/138 2022-11-27T16:46:16.824 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/138 2022-11-27T16:46:16.824 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/138 2022-11-27T16:46:16.825 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/138 2022-11-27T16:46:16.825 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/138 2022-11-27T16:46:16.825 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 72/138 2022-11-27T16:46:16.825 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 73/138 2022-11-27T16:46:16.826 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 74/138 2022-11-27T16:46:16.826 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/138 2022-11-27T16:46:16.826 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 76/138 2022-11-27T16:46:16.826 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 77/138 2022-11-27T16:46:16.826 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 78/138 2022-11-27T16:46:16.827 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 79/138 2022-11-27T16:46:16.827 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 80/138 2022-11-27T16:46:16.827 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 81/138 2022-11-27T16:46:16.827 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 82/138 2022-11-27T16:46:16.828 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 83/138 2022-11-27T16:46:16.828 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 84/138 2022-11-27T16:46:16.828 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/138 2022-11-27T16:46:16.828 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 86/138 2022-11-27T16:46:16.829 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/138 2022-11-27T16:46:16.829 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 88/138 2022-11-27T16:46:16.829 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 89/138 2022-11-27T16:46:16.829 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 90/138 2022-11-27T16:46:16.830 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 91/138 2022-11-27T16:46:16.830 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 92/138 2022-11-27T16:46:16.830 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 93/138 2022-11-27T16:46:16.830 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 94/138 2022-11-27T16:46:16.830 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 95/138 2022-11-27T16:46:16.831 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 96/138 2022-11-27T16:46:16.831 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/138 2022-11-27T16:46:16.831 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 98/138 2022-11-27T16:46:16.831 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : bison-3.0.4-10.el8.x86_64 99/138 2022-11-27T16:46:16.832 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 100/138 2022-11-27T16:46:16.832 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 101/138 2022-11-27T16:46:16.832 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 102/138 2022-11-27T16:46:16.832 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 103/138 2022-11-27T16:46:16.833 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 104/138 2022-11-27T16:46:16.833 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 105/138 2022-11-27T16:46:16.833 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 106/138 2022-11-27T16:46:16.834 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 107/138 2022-11-27T16:46:16.834 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 108/138 2022-11-27T16:46:16.835 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 109/138 2022-11-27T16:46:16.835 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 110/138 2022-11-27T16:46:16.836 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 111/138 2022-11-27T16:46:16.836 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 112/138 2022-11-27T16:46:16.836 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 113/138 2022-11-27T16:46:16.837 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 114/138 2022-11-27T16:46:16.837 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : flex-2.6.1-9.el8.x86_64 115/138 2022-11-27T16:46:16.838 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 116/138 2022-11-27T16:46:16.838 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : jq-1.6-3.el8.x86_64 117/138 2022-11-27T16:46:16.839 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 118/138 2022-11-27T16:46:16.839 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : automake-1.16.1-7.el8.noarch 119/138 2022-11-27T16:46:16.840 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 120/138 2022-11-27T16:46:16.840 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 121/138 2022-11-27T16:46:16.841 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 122/138 2022-11-27T16:46:16.841 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : libpq-13.5-1.el8.x86_64 123/138 2022-11-27T16:46:16.842 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 124/138 2022-11-27T16:46:16.842 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 125/138 2022-11-27T16:46:16.843 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 126/138 2022-11-27T16:46:16.843 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 127/138 2022-11-27T16:46:16.844 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 128/138 2022-11-27T16:46:16.844 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librados2-2:18.0.0-594.g697b0d99.el8.x86_64 129/138 2022-11-27T16:46:16.845 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/138 2022-11-27T16:46:16.845 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 131/138 2022-11-27T16:46:16.846 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/138 2022-11-27T16:46:16.846 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : openssl-1:1.1.1k-7.el8_6.x86_64 133/138 2022-11-27T16:46:16.846 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 134/138 2022-11-27T16:46:16.847 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : openssl-libs-1:1.1.1k-7.el8_6.x86_64 135/138 2022-11-27T16:46:16.847 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 136/138 2022-11-27T16:46:16.848 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 137/138 2022-11-27T16:46:17.239 INFO:teuthology.orchestra.run.smithi145.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:46:17.240 INFO:teuthology.orchestra.run.smithi145.stdout:Installed products updated. 2022-11-27T16:46:17.482 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:17.483 INFO:teuthology.orchestra.run.smithi145.stdout:Upgraded: 2022-11-27T16:46:17.483 INFO:teuthology.orchestra.run.smithi145.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T16:46:17.483 INFO:teuthology.orchestra.run.smithi145.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.483 INFO:teuthology.orchestra.run.smithi145.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.484 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:46:17.484 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:46:17.484 INFO:teuthology.orchestra.run.smithi145.stdout:Installed: 2022-11-27T16:46:17.484 INFO:teuthology.orchestra.run.smithi145.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T16:46:17.485 INFO:teuthology.orchestra.run.smithi145.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:46:17.485 INFO:teuthology.orchestra.run.smithi145.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:46:17.485 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.485 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.485 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.486 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.486 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.486 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.486 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.487 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.487 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.487 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.487 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.488 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.488 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.488 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.488 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.488 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.489 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.489 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.489 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.489 INFO:teuthology.orchestra.run.smithi145.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.490 INFO:teuthology.orchestra.run.smithi145.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.490 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.490 INFO:teuthology.orchestra.run.smithi145.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:17.490 INFO:teuthology.orchestra.run.smithi145.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T16:46:17.490 INFO:teuthology.orchestra.run.smithi145.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:46:17.491 INFO:teuthology.orchestra.run.smithi145.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T16:46:17.491 INFO:teuthology.orchestra.run.smithi145.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:46:17.491 INFO:teuthology.orchestra.run.smithi145.stdout: jq-1.6-3.el8.x86_64 2022-11-27T16:46:17.491 INFO:teuthology.orchestra.run.smithi145.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:46:17.492 INFO:teuthology.orchestra.run.smithi145.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T16:46:17.492 INFO:teuthology.orchestra.run.smithi145.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:46:17.492 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.493 INFO:teuthology.orchestra.run.smithi145.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.493 INFO:teuthology.orchestra.run.smithi145.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.493 INFO:teuthology.orchestra.run.smithi145.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T16:46:17.493 INFO:teuthology.orchestra.run.smithi145.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:46:17.493 INFO:teuthology.orchestra.run.smithi145.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:46:17.494 INFO:teuthology.orchestra.run.smithi145.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:46:17.494 INFO:teuthology.orchestra.run.smithi145.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:46:17.494 INFO:teuthology.orchestra.run.smithi145.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:46:17.494 INFO:teuthology.orchestra.run.smithi145.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.494 INFO:teuthology.orchestra.run.smithi145.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.495 INFO:teuthology.orchestra.run.smithi145.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:46:17.495 INFO:teuthology.orchestra.run.smithi145.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.496 INFO:teuthology.orchestra.run.smithi145.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T16:46:17.496 INFO:teuthology.orchestra.run.smithi145.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:46:17.496 INFO:teuthology.orchestra.run.smithi145.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:46:17.496 INFO:teuthology.orchestra.run.smithi145.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:46:17.497 INFO:teuthology.orchestra.run.smithi145.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T16:46:17.497 INFO:teuthology.orchestra.run.smithi145.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T16:46:17.497 INFO:teuthology.orchestra.run.smithi145.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:46:17.497 INFO:teuthology.orchestra.run.smithi145.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:46:17.498 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T16:46:17.498 INFO:teuthology.orchestra.run.smithi145.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T16:46:17.498 INFO:teuthology.orchestra.run.smithi145.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:46:17.498 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T16:46:17.499 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T16:46:17.499 INFO:teuthology.orchestra.run.smithi145.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T16:46:17.499 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:46:17.499 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:46:17.500 INFO:teuthology.orchestra.run.smithi145.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:46:17.500 INFO:teuthology.orchestra.run.smithi145.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:46:17.500 INFO:teuthology.orchestra.run.smithi145.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:46:17.500 INFO:teuthology.orchestra.run.smithi145.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:46:17.501 INFO:teuthology.orchestra.run.smithi145.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:46:17.501 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:46:17.501 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.501 INFO:teuthology.orchestra.run.smithi145.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.501 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.502 INFO:teuthology.orchestra.run.smithi145.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:46:17.502 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:46:17.502 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:46:17.502 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:46:17.503 INFO:teuthology.orchestra.run.smithi145.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:46:17.503 INFO:teuthology.orchestra.run.smithi145.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:46:17.503 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:46:17.503 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:46:17.504 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:46:17.504 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:46:17.504 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:46:17.504 INFO:teuthology.orchestra.run.smithi145.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:46:17.505 INFO:teuthology.orchestra.run.smithi145.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:46:17.505 INFO:teuthology.orchestra.run.smithi145.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:46:17.505 INFO:teuthology.orchestra.run.smithi145.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:46:17.505 INFO:teuthology.orchestra.run.smithi145.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:46:17.505 INFO:teuthology.orchestra.run.smithi145.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:46:17.506 INFO:teuthology.orchestra.run.smithi145.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:46:17.506 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:46:17.506 INFO:teuthology.orchestra.run.smithi145.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:46:17.506 INFO:teuthology.orchestra.run.smithi145.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:46:17.507 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:46:17.507 INFO:teuthology.orchestra.run.smithi145.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:46:17.507 INFO:teuthology.orchestra.run.smithi145.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:46:17.507 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:46:17.508 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:46:17.508 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:46:17.508 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:46:17.508 INFO:teuthology.orchestra.run.smithi145.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:46:17.508 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.509 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.509 INFO:teuthology.orchestra.run.smithi145.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:46:17.509 INFO:teuthology.orchestra.run.smithi145.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:46:17.509 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.510 INFO:teuthology.orchestra.run.smithi145.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:46:17.510 INFO:teuthology.orchestra.run.smithi145.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:46:17.510 INFO:teuthology.orchestra.run.smithi145.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T16:46:17.510 INFO:teuthology.orchestra.run.smithi145.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:46:17.510 INFO:teuthology.orchestra.run.smithi145.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:46:17.511 INFO:teuthology.orchestra.run.smithi145.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:46:17.511 INFO:teuthology.orchestra.run.smithi145.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:46:17.511 INFO:teuthology.orchestra.run.smithi145.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:46:17.511 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:46:17.512 INFO:teuthology.orchestra.run.smithi145.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:46:17.512 INFO:teuthology.orchestra.run.smithi145.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:46:17.512 INFO:teuthology.orchestra.run.smithi145.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:46:17.512 INFO:teuthology.orchestra.run.smithi145.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:46:17.513 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.513 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.513 INFO:teuthology.orchestra.run.smithi145.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:17.514 INFO:teuthology.orchestra.run.smithi145.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:46:17.514 INFO:teuthology.orchestra.run.smithi145.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:46:17.514 INFO:teuthology.orchestra.run.smithi145.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:46:17.514 INFO:teuthology.orchestra.run.smithi145.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:46:17.514 INFO:teuthology.orchestra.run.smithi145.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:46:17.515 INFO:teuthology.orchestra.run.smithi145.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T16:46:17.516 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:17.516 INFO:teuthology.orchestra.run.smithi145.stdout:Complete! 2022-11-27T16:46:17.662 DEBUG:teuthology.parallel:result is None 2022-11-27T16:46:18.678 INFO:teuthology.orchestra.run.smithi201.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 138/138 2022-11-27T16:46:18.679 INFO:teuthology.orchestra.run.smithi201.stdout:Installed products updated. 2022-11-27T16:46:18.908 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:18.908 INFO:teuthology.orchestra.run.smithi201.stdout:Upgraded: 2022-11-27T16:46:18.909 INFO:teuthology.orchestra.run.smithi201.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2022-11-27T16:46:18.909 INFO:teuthology.orchestra.run.smithi201.stdout: librados2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.909 INFO:teuthology.orchestra.run.smithi201.stdout: librbd1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.909 INFO:teuthology.orchestra.run.smithi201.stdout: openssl-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:46:18.909 INFO:teuthology.orchestra.run.smithi201.stdout: openssl-libs-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:46:18.910 INFO:teuthology.orchestra.run.smithi201.stdout:Installed: 2022-11-27T16:46:18.910 INFO:teuthology.orchestra.run.smithi201.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2022-11-27T16:46:18.910 INFO:teuthology.orchestra.run.smithi201.stdout: automake-1.16.1-7.el8.noarch 2022-11-27T16:46:18.910 INFO:teuthology.orchestra.run.smithi201.stdout: bison-3.0.4-10.el8.x86_64 2022-11-27T16:46:18.911 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.911 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-base-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.911 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.911 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.912 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-grafana-dashboards-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.912 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-immutable-object-cache-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.912 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mds-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.912 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.912 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.913 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-dashboard-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.913 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-diskprediction-local-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.913 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-modules-core-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.913 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mgr-rook-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.914 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-mon-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.914 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-osd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.914 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-prometheus-alerts-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.914 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-radosgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.914 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-selinux-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.915 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-test-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.915 INFO:teuthology.orchestra.run.smithi201.stdout: ceph-volume-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.915 INFO:teuthology.orchestra.run.smithi201.stdout: cephadm-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.915 INFO:teuthology.orchestra.run.smithi201.stdout: cephfs-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.916 INFO:teuthology.orchestra.run.smithi201.stdout: cephfs-top-2:18.0.0-594.g697b0d99.el8.noarch 2022-11-27T16:46:18.916 INFO:teuthology.orchestra.run.smithi201.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2022-11-27T16:46:18.916 INFO:teuthology.orchestra.run.smithi201.stdout: flex-2.6.1-9.el8.x86_64 2022-11-27T16:46:18.916 INFO:teuthology.orchestra.run.smithi201.stdout: fuse-2.9.7-15.el8.x86_64 2022-11-27T16:46:18.917 INFO:teuthology.orchestra.run.smithi201.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2022-11-27T16:46:18.917 INFO:teuthology.orchestra.run.smithi201.stdout: jq-1.6-3.el8.x86_64 2022-11-27T16:46:18.917 INFO:teuthology.orchestra.run.smithi201.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2022-11-27T16:46:18.917 INFO:teuthology.orchestra.run.smithi201.stdout: krb5-devel-1.18.2-14.el8.x86_64 2022-11-27T16:46:18.917 INFO:teuthology.orchestra.run.smithi201.stdout: libaio-devel-0.3.112-1.el8.x86_64 2022-11-27T16:46:18.918 INFO:teuthology.orchestra.run.smithi201.stdout: libcephfs-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.918 INFO:teuthology.orchestra.run.smithi201.stdout: libcephfs2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.918 INFO:teuthology.orchestra.run.smithi201.stdout: libcephsqlite-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.918 INFO:teuthology.orchestra.run.smithi201.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2022-11-27T16:46:18.918 INFO:teuthology.orchestra.run.smithi201.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:46:18.919 INFO:teuthology.orchestra.run.smithi201.stdout: liboath-2.6.2-3.el8.x86_64 2022-11-27T16:46:18.919 INFO:teuthology.orchestra.run.smithi201.stdout: libpq-13.5-1.el8.x86_64 2022-11-27T16:46:18.919 INFO:teuthology.orchestra.run.smithi201.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2022-11-27T16:46:18.919 INFO:teuthology.orchestra.run.smithi201.stdout: librabbitmq-0.9.0-3.el8.x86_64 2022-11-27T16:46:18.920 INFO:teuthology.orchestra.run.smithi201.stdout: librados-devel-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.920 INFO:teuthology.orchestra.run.smithi201.stdout: libradosstriper1-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.920 INFO:teuthology.orchestra.run.smithi201.stdout: librdkafka-0.11.4-3.el8.x86_64 2022-11-27T16:46:18.920 INFO:teuthology.orchestra.run.smithi201.stdout: librgw2-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.920 INFO:teuthology.orchestra.run.smithi201.stdout: libselinux-devel-2.9-5.el8.x86_64 2022-11-27T16:46:18.921 INFO:teuthology.orchestra.run.smithi201.stdout: libsepol-devel-2.9-3.el8.x86_64 2022-11-27T16:46:18.921 INFO:teuthology.orchestra.run.smithi201.stdout: libtool-2.4.6-25.el8.x86_64 2022-11-27T16:46:18.921 INFO:teuthology.orchestra.run.smithi201.stdout: libunwind-1.3.1-3.el8.x86_64 2022-11-27T16:46:18.921 INFO:teuthology.orchestra.run.smithi201.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2022-11-27T16:46:18.922 INFO:teuthology.orchestra.run.smithi201.stdout: libverto-devel-0.3.0-5.el8.x86_64 2022-11-27T16:46:18.922 INFO:teuthology.orchestra.run.smithi201.stdout: lttng-ust-2.8.1-11.el8.x86_64 2022-11-27T16:46:18.922 INFO:teuthology.orchestra.run.smithi201.stdout: oniguruma-6.8.2-2.el8.x86_64 2022-11-27T16:46:18.922 INFO:teuthology.orchestra.run.smithi201.stdout: openblas-0.3.15-3.el8.x86_64 2022-11-27T16:46:18.922 INFO:teuthology.orchestra.run.smithi201.stdout: openblas-threads-0.3.15-3.el8.x86_64 2022-11-27T16:46:18.923 INFO:teuthology.orchestra.run.smithi201.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2022-11-27T16:46:18.923 INFO:teuthology.orchestra.run.smithi201.stdout: pcre2-devel-10.32-2.el8.x86_64 2022-11-27T16:46:18.923 INFO:teuthology.orchestra.run.smithi201.stdout: pcre2-utf16-10.32-2.el8.x86_64 2022-11-27T16:46:18.923 INFO:teuthology.orchestra.run.smithi201.stdout: pcre2-utf32-10.32-2.el8.x86_64 2022-11-27T16:46:18.924 INFO:teuthology.orchestra.run.smithi201.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:46:18.924 INFO:teuthology.orchestra.run.smithi201.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:46:18.924 INFO:teuthology.orchestra.run.smithi201.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2022-11-27T16:46:18.924 INFO:teuthology.orchestra.run.smithi201.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2022-11-27T16:46:18.924 INFO:teuthology.orchestra.run.smithi201.stdout: python3-babel-2.5.1-7.el8.noarch 2022-11-27T16:46:18.925 INFO:teuthology.orchestra.run.smithi201.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2022-11-27T16:46:18.925 INFO:teuthology.orchestra.run.smithi201.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2022-11-27T16:46:18.925 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cachetools-3.1.1-4.el8.noarch 2022-11-27T16:46:18.925 INFO:teuthology.orchestra.run.smithi201.stdout: python3-ceph-argparse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.926 INFO:teuthology.orchestra.run.smithi201.stdout: python3-ceph-common-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.926 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cephfs-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.926 INFO:teuthology.orchestra.run.smithi201.stdout: python3-certifi-2018.10.15-7.el8.noarch 2022-11-27T16:46:18.926 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cffi-1.11.5-5.el8.x86_64 2022-11-27T16:46:18.926 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cheroot-8.5.2-1.el8.noarch 2022-11-27T16:46:18.927 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2022-11-27T16:46:18.927 INFO:teuthology.orchestra.run.smithi201.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2022-11-27T16:46:18.927 INFO:teuthology.orchestra.run.smithi201.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2022-11-27T16:46:18.928 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jaraco-6.2-6.el8.noarch 2022-11-27T16:46:18.928 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2022-11-27T16:46:18.928 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jinja2-2.10.1-3.el8.noarch 2022-11-27T16:46:18.928 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jsonpatch-1.21-2.el8.noarch 2022-11-27T16:46:18.928 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jsonpointer-1.10-11.el8.noarch 2022-11-27T16:46:18.929 INFO:teuthology.orchestra.run.smithi201.stdout: python3-jwt-1.6.1-2.el8.noarch 2022-11-27T16:46:18.929 INFO:teuthology.orchestra.run.smithi201.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2022-11-27T16:46:18.929 INFO:teuthology.orchestra.run.smithi201.stdout: python3-logutils-0.3.5-11.el8.noarch 2022-11-27T16:46:18.929 INFO:teuthology.orchestra.run.smithi201.stdout: python3-mako-1.0.6-13.el8.noarch 2022-11-27T16:46:18.930 INFO:teuthology.orchestra.run.smithi201.stdout: python3-markupsafe-0.23-19.el8.x86_64 2022-11-27T16:46:18.930 INFO:teuthology.orchestra.run.smithi201.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2022-11-27T16:46:18.930 INFO:teuthology.orchestra.run.smithi201.stdout: python3-natsort-7.1.1-2.el8.noarch 2022-11-27T16:46:18.930 INFO:teuthology.orchestra.run.smithi201.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2022-11-27T16:46:18.930 INFO:teuthology.orchestra.run.smithi201.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2022-11-27T16:46:18.931 INFO:teuthology.orchestra.run.smithi201.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2022-11-27T16:46:18.931 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pecan-1.3.2-9.el8.noarch 2022-11-27T16:46:18.931 INFO:teuthology.orchestra.run.smithi201.stdout: python3-portend-2.6-1.el8.noarch 2022-11-27T16:46:18.932 INFO:teuthology.orchestra.run.smithi201.stdout: python3-prettytable-0.7.2-14.el8.noarch 2022-11-27T16:46:18.932 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2022-11-27T16:46:18.932 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2022-11-27T16:46:18.933 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2022-11-27T16:46:18.933 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pycparser-2.14-14.el8.noarch 2022-11-27T16:46:18.933 INFO:teuthology.orchestra.run.smithi201.stdout: python3-pytz-2017.2-9.el8.noarch 2022-11-27T16:46:18.933 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rados-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.934 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.934 INFO:teuthology.orchestra.run.smithi201.stdout: python3-repoze-lru-0.7-6.el8.noarch 2022-11-27T16:46:18.934 INFO:teuthology.orchestra.run.smithi201.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2022-11-27T16:46:18.934 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rgw-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.934 INFO:teuthology.orchestra.run.smithi201.stdout: python3-routes-2.4.1-12.el8.noarch 2022-11-27T16:46:18.935 INFO:teuthology.orchestra.run.smithi201.stdout: python3-rsa-4.8-1.el8.noarch 2022-11-27T16:46:18.935 INFO:teuthology.orchestra.run.smithi201.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2022-11-27T16:46:18.935 INFO:teuthology.orchestra.run.smithi201.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2022-11-27T16:46:18.935 INFO:teuthology.orchestra.run.smithi201.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2022-11-27T16:46:18.936 INFO:teuthology.orchestra.run.smithi201.stdout: python3-tempora-1.14.1-5.el8.noarch 2022-11-27T16:46:18.936 INFO:teuthology.orchestra.run.smithi201.stdout: python3-trustme-0.6.0-4.el8.noarch 2022-11-27T16:46:18.936 INFO:teuthology.orchestra.run.smithi201.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2022-11-27T16:46:18.936 INFO:teuthology.orchestra.run.smithi201.stdout: python3-webob-1.8.5-1.el8.1.noarch 2022-11-27T16:46:18.937 INFO:teuthology.orchestra.run.smithi201.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2022-11-27T16:46:18.937 INFO:teuthology.orchestra.run.smithi201.stdout: python3-webtest-2.0.33-1.el8.noarch 2022-11-27T16:46:18.938 INFO:teuthology.orchestra.run.smithi201.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2022-11-27T16:46:18.938 INFO:teuthology.orchestra.run.smithi201.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2022-11-27T16:46:18.939 INFO:teuthology.orchestra.run.smithi201.stdout: rbd-fuse-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.939 INFO:teuthology.orchestra.run.smithi201.stdout: rbd-mirror-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.940 INFO:teuthology.orchestra.run.smithi201.stdout: rbd-nbd-2:18.0.0-594.g697b0d99.el8.x86_64 2022-11-27T16:46:18.940 INFO:teuthology.orchestra.run.smithi201.stdout: socat-1.7.4.1-1.el8.x86_64 2022-11-27T16:46:18.941 INFO:teuthology.orchestra.run.smithi201.stdout: thrift-0.13.0-2.el8.x86_64 2022-11-27T16:46:18.941 INFO:teuthology.orchestra.run.smithi201.stdout: uuid-1.6.2-43.el8.x86_64 2022-11-27T16:46:18.941 INFO:teuthology.orchestra.run.smithi201.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2022-11-27T16:46:18.942 INFO:teuthology.orchestra.run.smithi201.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2022-11-27T16:46:18.943 INFO:teuthology.orchestra.run.smithi201.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2022-11-27T16:46:18.943 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:18.944 INFO:teuthology.orchestra.run.smithi201.stdout:Complete! 2022-11-27T16:46:19.096 DEBUG:teuthology.parallel:result is None 2022-11-27T16:46:19.097 DEBUG:teuthology.orchestra.run.smithi145:> sudo lsb_release -is 2022-11-27T16:46:19.157 INFO:teuthology.orchestra.run.smithi145.stdout:RedHatEnterprise 2022-11-27T16:46:19.158 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:46:19.159 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:46:19.326 DEBUG:teuthology.orchestra.run.smithi145:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:46:19.349 INFO:teuthology.orchestra.run.smithi145.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:46:19.350 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:46:19.350 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:46:19.351 DEBUG:teuthology.orchestra.run.smithi201:> sudo lsb_release -is 2022-11-27T16:46:19.408 INFO:teuthology.orchestra.run.smithi201.stdout:RedHatEnterprise 2022-11-27T16:46:19.410 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2022-11-27T16:46:19.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=697b0d99d9a52df92b20c8e053dcd46c75f364a9 2022-11-27T16:46:19.582 DEBUG:teuthology.orchestra.run.smithi201:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2022-11-27T16:46:19.604 INFO:teuthology.orchestra.run.smithi201.stdout:18.0.0-594.g697b0d99.el8 2022-11-27T16:46:19.604 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-594.g697b0d99.el8 2022-11-27T16:46:19.605 INFO:teuthology.task.install:The correct ceph version 18.0.0-594.g697b0d99 is installed. 2022-11-27T16:46:19.606 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2022-11-27T16:46:19.606 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:19.607 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:46:19.635 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:19.636 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2022-11-27T16:46:19.672 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2022-11-27T16:46:19.673 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:19.673 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:46:19.701 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:46:19.769 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:19.770 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd of=/usr/bin/daemon-helper 2022-11-27T16:46:19.797 DEBUG:teuthology.orchestra.run.smithi201:> sudo chmod a=rx -- /usr/bin/daemon-helper 2022-11-27T16:46:19.864 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2022-11-27T16:46:19.865 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:19.865 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:46:19.892 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:46:19.957 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:19.958 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd of=/usr/bin/adjust-ulimits 2022-11-27T16:46:19.984 DEBUG:teuthology.orchestra.run.smithi201:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2022-11-27T16:46:20.051 INFO:teuthology.run_tasks:Running task ceph... 2022-11-27T16:46:20.137 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2022-11-27T16:46:20.138 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 777 /var/log/ceph 2022-11-27T16:46:20.141 DEBUG:teuthology.orchestra.run.smithi201:> sudo chmod 777 /var/log/ceph 2022-11-27T16:46:20.167 INFO:tasks.ceph:Disabling ceph logrotate... 2022-11-27T16:46:20.168 DEBUG:teuthology.orchestra.run.smithi145:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:46:20.209 DEBUG:teuthology.orchestra.run.smithi201:> sudo rm -f -- /etc/logrotate.d/ceph 2022-11-27T16:46:20.237 INFO:tasks.ceph:Creating extra log directories... 2022-11-27T16:46:20.238 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:46:20.274 DEBUG:teuthology.orchestra.run.smithi201:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2022-11-27T16:46:20.316 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2022-11-27T16:46:20.317 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2022-11-27T16:46:20.317 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2022-11-27T16:46:20.318 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:20.318 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:46:20.356 DEBUG:teuthology.orchestra.run.smithi145:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:46:20.472 DEBUG:teuthology.orchestra.run.smithi145:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:46:20.499 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:20.499 DEBUG:teuthology.orchestra.run.smithi201:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2022-11-27T16:46:20.522 DEBUG:teuthology.orchestra.run.smithi201:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2022-11-27T16:46:20.636 DEBUG:teuthology.orchestra.run.smithi201:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2022-11-27T16:46:20.668 INFO:tasks.ceph:Creating ceph cluster ceph... 2022-11-27T16:46:20.668 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2022-11-27T16:46:20.669 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi/7088670', 'branch': 'main', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'ceph-qa@ceph.io', 'first_in_suite': False, 'job_id': '7088670', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2022-11-23_17:24:26-fs-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_697b0d99d9a52df92b20c8e053dcd46c75f364a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi145.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBADicDbkWTVsl36h8yPnxOjPpkZf/nawuzP7dcV4oZRM/tv9ZZtofvr3r2c1FfJzaNfs+5SvEvk7FI4m6D/h61g=', 'smithi201.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1x/o0GmT7+k5M2rxYh1XiqiIlBd1hAaY6pwKvVnXg7SkCMMzy1Y9/OPMz1gFSTk9iEJXvl/eICGLjSs67ZLcSFvltC+ztPFDFMQFtoibLRbppON8RvKGGIYGurq70wEZWufMFI4e2iVTL5Ha38bxNGAGY8fRbAuFFQPVqDxXHo6IFYnIkRA0fztraW/Ic9sWeDyZdAbUmm4kR83SD07BFx3Noj0XGO1/uxCvfASolmH7nnhldWSLWblZ0puTmPLDDAMLVAGpO1438MecwP4kPcJx+UJxLpHYaZZEhX0bD95p4fWqLV1uueZT6FGhAQFMUn9krUCrw6Z9EicYYHnXpMH4iSlvZgMtr9+z+oj6ygpCHD5JlKTZyZfzoWJzPQ0m2AGZs0nfvFgJ2DEynNBSgQhTrcqncnPekC/9tkH63DdaN/Vl0D3sFo+yJda36HnCco323xXZcQMGKxoUCTFpNPI1ycv9EcgsTvENnzlaU5ISJjcme8hOqNrxEHkamg38='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '697b0d99d9a52df92b20c8e053dcd46c75f364a9'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '4da97cf64e542f347ec47b7bdbe5eca99759f9b7', 'timestamp': '2022-11-23_17:24:26', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.20399'} 2022-11-27T16:46:20.670 DEBUG:teuthology.orchestra.run.smithi145:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:46:20.672 DEBUG:teuthology.orchestra.run.smithi201:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2022-11-27T16:46:20.719 DEBUG:teuthology.orchestra.run.smithi145:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:46:20.730 DEBUG:teuthology.orchestra.run.smithi201:> sudo install -d -m0777 -- /var/run/ceph 2022-11-27T16:46:20.786 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:20.786 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:46:20.811 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:46:20.812 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_1 2022-11-27T16:46:20.868 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:46:20.869 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:20.869 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 363992 Links: 1 2022-11-27T16:46:20.869 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:20.869 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:20.870 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2022-11-27 16:46:14.994194732 +0000 2022-11-27T16:46:20.870 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2022-11-27 16:43:27.840959593 +0000 2022-11-27T16:46:20.870 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2022-11-27 16:43:27.840959593 +0000 2022-11-27T16:46:20.871 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2022-11-27T16:46:20.871 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:46:20.933 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2022-11-27T16:46:20.933 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2022-11-27T16:46:20.934 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000265462 s, 1.9 MB/s 2022-11-27T16:46:20.935 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:46:20.993 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_2 2022-11-27T16:46:21.051 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:46:21.051 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:21.052 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 354217 Links: 1 2022-11-27T16:46:21.052 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:21.053 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:21.053 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2022-11-27 16:46:14.994194732 +0000 2022-11-27T16:46:21.053 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2022-11-27 16:43:28.389948020 +0000 2022-11-27T16:46:21.054 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2022-11-27 16:43:28.389948020 +0000 2022-11-27T16:46:21.054 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2022-11-27T16:46:21.055 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:46:21.116 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2022-11-27T16:46:21.117 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2022-11-27T16:46:21.117 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000204854 s, 2.5 MB/s 2022-11-27T16:46:21.119 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:46:21.177 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_3 2022-11-27T16:46:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:46:21.235 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:21.235 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 360356 Links: 1 2022-11-27T16:46:21.235 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:21.236 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:21.236 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2022-11-27 16:46:14.994194732 +0000 2022-11-27T16:46:21.236 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2022-11-27 16:43:28.934936531 +0000 2022-11-27T16:46:21.236 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2022-11-27 16:43:28.934936531 +0000 2022-11-27T16:46:21.237 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2022-11-27T16:46:21.237 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:46:21.300 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2022-11-27T16:46:21.300 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2022-11-27T16:46:21.300 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000192575 s, 2.7 MB/s 2022-11-27T16:46:21.301 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:46:21.359 DEBUG:teuthology.orchestra.run.smithi145:> stat /dev/vg_nvme/lv_4 2022-11-27T16:46:21.416 INFO:teuthology.orchestra.run.smithi145.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:46:21.417 INFO:teuthology.orchestra.run.smithi145.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:21.417 INFO:teuthology.orchestra.run.smithi145.stdout:Device: 6h/6d Inode: 367873 Links: 1 2022-11-27T16:46:21.417 INFO:teuthology.orchestra.run.smithi145.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:21.418 INFO:teuthology.orchestra.run.smithi145.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:21.418 INFO:teuthology.orchestra.run.smithi145.stdout:Access: 2022-11-27 16:46:14.994194732 +0000 2022-11-27T16:46:21.418 INFO:teuthology.orchestra.run.smithi145.stdout:Modify: 2022-11-27 16:43:29.469925253 +0000 2022-11-27T16:46:21.418 INFO:teuthology.orchestra.run.smithi145.stdout:Change: 2022-11-27 16:43:29.469925253 +0000 2022-11-27T16:46:21.419 INFO:teuthology.orchestra.run.smithi145.stdout: Birth: - 2022-11-27T16:46:21.419 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:46:21.481 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records in 2022-11-27T16:46:21.482 INFO:teuthology.orchestra.run.smithi145.stderr:1+0 records out 2022-11-27T16:46:21.482 INFO:teuthology.orchestra.run.smithi145.stderr:512 bytes copied, 0.000243687 s, 2.1 MB/s 2022-11-27T16:46:21.483 DEBUG:teuthology.orchestra.run.smithi145:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:46:21.542 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:21.543 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:21.543 DEBUG:teuthology.orchestra.run.smithi201:> dd if=/scratch_devs of=/dev/stdout 2022-11-27T16:46:21.565 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2022-11-27T16:46:21.566 DEBUG:teuthology.orchestra.run.smithi201:> stat /dev/vg_nvme/lv_1 2022-11-27T16:46:21.624 INFO:teuthology.orchestra.run.smithi201.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2022-11-27T16:46:21.624 INFO:teuthology.orchestra.run.smithi201.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:21.624 INFO:teuthology.orchestra.run.smithi201.stdout:Device: 6h/6d Inode: 367773 Links: 1 2022-11-27T16:46:21.625 INFO:teuthology.orchestra.run.smithi201.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:21.625 INFO:teuthology.orchestra.run.smithi201.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:21.625 INFO:teuthology.orchestra.run.smithi201.stdout:Access: 2022-11-27 16:46:16.461502666 +0000 2022-11-27T16:46:21.625 INFO:teuthology.orchestra.run.smithi201.stdout:Modify: 2022-11-27 16:43:28.858886518 +0000 2022-11-27T16:46:21.626 INFO:teuthology.orchestra.run.smithi201.stdout:Change: 2022-11-27 16:43:28.858886518 +0000 2022-11-27T16:46:21.626 INFO:teuthology.orchestra.run.smithi201.stdout: Birth: - 2022-11-27T16:46:21.626 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2022-11-27T16:46:21.691 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records in 2022-11-27T16:46:21.692 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records out 2022-11-27T16:46:21.692 INFO:teuthology.orchestra.run.smithi201.stderr:512 bytes copied, 0.000203497 s, 2.5 MB/s 2022-11-27T16:46:21.693 DEBUG:teuthology.orchestra.run.smithi201:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2022-11-27T16:46:21.752 DEBUG:teuthology.orchestra.run.smithi201:> stat /dev/vg_nvme/lv_2 2022-11-27T16:46:21.810 INFO:teuthology.orchestra.run.smithi201.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2022-11-27T16:46:21.810 INFO:teuthology.orchestra.run.smithi201.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:21.810 INFO:teuthology.orchestra.run.smithi201.stdout:Device: 6h/6d Inode: 369688 Links: 1 2022-11-27T16:46:21.811 INFO:teuthology.orchestra.run.smithi201.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:21.811 INFO:teuthology.orchestra.run.smithi201.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:21.811 INFO:teuthology.orchestra.run.smithi201.stdout:Access: 2022-11-27 16:46:16.461502666 +0000 2022-11-27T16:46:21.811 INFO:teuthology.orchestra.run.smithi201.stdout:Modify: 2022-11-27 16:43:29.406878165 +0000 2022-11-27T16:46:21.811 INFO:teuthology.orchestra.run.smithi201.stdout:Change: 2022-11-27 16:43:29.406878165 +0000 2022-11-27T16:46:21.812 INFO:teuthology.orchestra.run.smithi201.stdout: Birth: - 2022-11-27T16:46:21.812 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2022-11-27T16:46:21.876 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records in 2022-11-27T16:46:21.877 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records out 2022-11-27T16:46:21.877 INFO:teuthology.orchestra.run.smithi201.stderr:512 bytes copied, 0.000212314 s, 2.4 MB/s 2022-11-27T16:46:21.879 DEBUG:teuthology.orchestra.run.smithi201:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2022-11-27T16:46:21.937 DEBUG:teuthology.orchestra.run.smithi201:> stat /dev/vg_nvme/lv_3 2022-11-27T16:46:21.996 INFO:teuthology.orchestra.run.smithi201.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2022-11-27T16:46:21.997 INFO:teuthology.orchestra.run.smithi201.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:21.997 INFO:teuthology.orchestra.run.smithi201.stdout:Device: 6h/6d Inode: 367959 Links: 1 2022-11-27T16:46:21.998 INFO:teuthology.orchestra.run.smithi201.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:21.998 INFO:teuthology.orchestra.run.smithi201.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:21.999 INFO:teuthology.orchestra.run.smithi201.stdout:Access: 2022-11-27 16:46:16.461502666 +0000 2022-11-27T16:46:21.999 INFO:teuthology.orchestra.run.smithi201.stdout:Modify: 2022-11-27 16:43:29.942869995 +0000 2022-11-27T16:46:21.999 INFO:teuthology.orchestra.run.smithi201.stdout:Change: 2022-11-27 16:43:29.942869995 +0000 2022-11-27T16:46:21.999 INFO:teuthology.orchestra.run.smithi201.stdout: Birth: - 2022-11-27T16:46:22.000 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2022-11-27T16:46:22.062 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records in 2022-11-27T16:46:22.062 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records out 2022-11-27T16:46:22.063 INFO:teuthology.orchestra.run.smithi201.stderr:512 bytes copied, 0.000210795 s, 2.4 MB/s 2022-11-27T16:46:22.064 DEBUG:teuthology.orchestra.run.smithi201:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2022-11-27T16:46:22.122 DEBUG:teuthology.orchestra.run.smithi201:> stat /dev/vg_nvme/lv_4 2022-11-27T16:46:22.179 INFO:teuthology.orchestra.run.smithi201.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2022-11-27T16:46:22.180 INFO:teuthology.orchestra.run.smithi201.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2022-11-27T16:46:22.180 INFO:teuthology.orchestra.run.smithi201.stdout:Device: 6h/6d Inode: 365018 Links: 1 2022-11-27T16:46:22.180 INFO:teuthology.orchestra.run.smithi201.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2022-11-27T16:46:22.180 INFO:teuthology.orchestra.run.smithi201.stdout:Context: system_u:object_r:device_t:s0 2022-11-27T16:46:22.181 INFO:teuthology.orchestra.run.smithi201.stdout:Access: 2022-11-27 16:46:16.461502666 +0000 2022-11-27T16:46:22.181 INFO:teuthology.orchestra.run.smithi201.stdout:Modify: 2022-11-27 16:43:30.482861764 +0000 2022-11-27T16:46:22.181 INFO:teuthology.orchestra.run.smithi201.stdout:Change: 2022-11-27 16:43:30.482861764 +0000 2022-11-27T16:46:22.181 INFO:teuthology.orchestra.run.smithi201.stdout: Birth: - 2022-11-27T16:46:22.182 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2022-11-27T16:46:22.244 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records in 2022-11-27T16:46:22.244 INFO:teuthology.orchestra.run.smithi201.stderr:1+0 records out 2022-11-27T16:46:22.245 INFO:teuthology.orchestra.run.smithi201.stderr:512 bytes copied, 0.000294483 s, 1.7 MB/s 2022-11-27T16:46:22.246 DEBUG:teuthology.orchestra.run.smithi201:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2022-11-27T16:46:22.304 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:22.378 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi201.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:46:22.378 INFO:tasks.ceph:Generating config... 2022-11-27T16:46:22.380 INFO:tasks.ceph:[client] client mount timeout = 600 2022-11-27T16:46:22.380 INFO:tasks.ceph:[client] debug client = 20 2022-11-27T16:46:22.380 INFO:tasks.ceph:[client] debug ms = 1 2022-11-27T16:46:22.381 INFO:tasks.ceph:[client] pid file = 2022-11-27T16:46:22.381 INFO:tasks.ceph:[client] rados mon op timeout = 900 2022-11-27T16:46:22.381 INFO:tasks.ceph:[client] rados osd op timeout = 900 2022-11-27T16:46:22.382 INFO:tasks.ceph:[client.0] debug client = 20 2022-11-27T16:46:22.382 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2022-11-27T16:46:22.382 INFO:tasks.ceph:[client.0] debug objecter = 20 2022-11-27T16:46:22.383 INFO:tasks.ceph:[client.1] debug client = 20 2022-11-27T16:46:22.383 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2022-11-27T16:46:22.383 INFO:tasks.ceph:[client.1] debug objecter = 20 2022-11-27T16:46:22.384 INFO:tasks.ceph:[global] ms die on skipped message = False 2022-11-27T16:46:22.384 INFO:tasks.ceph:[mds] debug mds = 20 2022-11-27T16:46:22.384 INFO:tasks.ceph:[mds] debug mds balancer = 20 2022-11-27T16:46:22.385 INFO:tasks.ceph:[mds] debug ms = 1 2022-11-27T16:46:22.385 INFO:tasks.ceph:[mds] mds debug frag = True 2022-11-27T16:46:22.385 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2022-11-27T16:46:22.386 INFO:tasks.ceph:[mds] mds op complaint time = 180 2022-11-27T16:46:22.386 INFO:tasks.ceph:[mds] mds verify scatter = True 2022-11-27T16:46:22.386 INFO:tasks.ceph:[mds] osd op complaint time = 180 2022-11-27T16:46:22.386 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2022-11-27T16:46:22.387 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2022-11-27T16:46:22.387 INFO:tasks.ceph:[mgr] debug mgr = 20 2022-11-27T16:46:22.388 INFO:tasks.ceph:[mgr] debug ms = 1 2022-11-27T16:46:22.389 INFO:tasks.ceph:[mon] debug mon = 20 2022-11-27T16:46:22.389 INFO:tasks.ceph:[mon] debug ms = 1 2022-11-27T16:46:22.390 INFO:tasks.ceph:[mon] debug paxos = 20 2022-11-27T16:46:22.390 INFO:tasks.ceph:[mon] mon op complaint time = 120 2022-11-27T16:46:22.391 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2022-11-27T16:46:22.391 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2022-11-27T16:46:22.391 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2022-11-27T16:46:22.392 INFO:tasks.ceph:[osd] bdev async discard = True 2022-11-27T16:46:22.392 INFO:tasks.ceph:[osd] bdev enable discard = True 2022-11-27T16:46:22.392 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2022-11-27T16:46:22.393 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2022-11-27T16:46:22.393 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2022-11-27T16:46:22.393 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2022-11-27T16:46:22.394 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2022-11-27T16:46:22.394 INFO:tasks.ceph:[osd] debug ms = 1 2022-11-27T16:46:22.395 INFO:tasks.ceph:[osd] debug osd = 20 2022-11-27T16:46:22.395 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2022-11-27T16:46:22.395 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2022-11-27T16:46:22.395 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2022-11-27T16:46:22.396 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2022-11-27T16:46:22.396 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2022-11-27T16:46:22.396 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2022-11-27T16:46:22.397 INFO:tasks.ceph:[osd] osd mon report interval = 5 2022-11-27T16:46:22.397 INFO:tasks.ceph:[osd] osd objectstore = memstore 2022-11-27T16:46:22.397 INFO:tasks.ceph:[osd] osd op complaint time = 180 2022-11-27T16:46:22.398 INFO:tasks.ceph:Setting up mon.a... 2022-11-27T16:46:22.398 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2022-11-27T16:46:22.457 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.keyring 2022-11-27T16:46:22.461 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2022-11-27T16:46:22.510 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:46:22.538 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.145'), ('mon.b', '172.21.15.201'), ('mon.c', '[v2:172.21.15.201:3301,v1:172.21.15.201:6790]')] 2022-11-27T16:46:22.539 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.145,172.21.15.201,[v2:172.21.15.201:3301,v1:172.21.15.201:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2022-11-27T16:46:22.540 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:22.541 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:46:22.595 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.145 --add b 172.21.15.201 --addv c '[v2:172.21.15.201:3301,v1:172.21.15.201:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:22.683 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:22.683 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: generated fsid 0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7 2022-11-27T16:46:22.683 INFO:teuthology.orchestra.run.smithi145.stdout:setting min_mon_release = pacific 2022-11-27T16:46:22.684 INFO:teuthology.orchestra.run.smithi145.stdout:epoch 0 2022-11-27T16:46:22.685 INFO:teuthology.orchestra.run.smithi145.stdout:fsid 0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7 2022-11-27T16:46:22.685 INFO:teuthology.orchestra.run.smithi145.stdout:last_changed 2022-11-27T16:46:22.682111+0000 2022-11-27T16:46:22.685 INFO:teuthology.orchestra.run.smithi145.stdout:created 2022-11-27T16:46:22.682111+0000 2022-11-27T16:46:22.686 INFO:teuthology.orchestra.run.smithi145.stdout:min_mon_release 16 (pacific) 2022-11-27T16:46:22.686 INFO:teuthology.orchestra.run.smithi145.stdout:election_strategy: 1 2022-11-27T16:46:22.686 INFO:teuthology.orchestra.run.smithi145.stdout:0: [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] mon.a 2022-11-27T16:46:22.686 INFO:teuthology.orchestra.run.smithi145.stdout:1: [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] mon.b 2022-11-27T16:46:22.687 INFO:teuthology.orchestra.run.smithi145.stdout:2: [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] mon.c 2022-11-27T16:46:22.687 INFO:teuthology.orchestra.run.smithi145.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2022-11-27T16:46:22.688 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2022-11-27T16:46:22.705 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7... 2022-11-27T16:46:22.706 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:46:22.748 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2022-11-27T16:46:22.790 INFO:teuthology.orchestra.run.smithi145.stdout:[global] 2022-11-27T16:46:22.791 INFO:teuthology.orchestra.run.smithi145.stdout: chdir = "" 2022-11-27T16:46:22.792 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:46:22.792 INFO:teuthology.orchestra.run.smithi145.stdout: auth supported = cephx 2022-11-27T16:46:22.792 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.793 INFO:teuthology.orchestra.run.smithi145.stdout: filestore xattr use omap = true 2022-11-27T16:46:22.793 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.794 INFO:teuthology.orchestra.run.smithi145.stdout: mon clock drift allowed = 1.000 2022-11-27T16:46:22.794 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.795 INFO:teuthology.orchestra.run.smithi145.stdout: osd crush chooseleaf type = 0 2022-11-27T16:46:22.795 INFO:teuthology.orchestra.run.smithi145.stdout: auth debug = true 2022-11-27T16:46:22.796 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.796 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on old message = true 2022-11-27T16:46:22.796 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on bug = true 2022-11-27T16:46:22.797 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.797 INFO:teuthology.orchestra.run.smithi145.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:46:22.798 INFO:teuthology.orchestra.run.smithi145.stdout: mon pg warn max object skew = 0 2022-11-27T16:46:22.799 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.799 INFO:teuthology.orchestra.run.smithi145.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:46:22.799 INFO:teuthology.orchestra.run.smithi145.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:46:22.800 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.800 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default size = 2 2022-11-27T16:46:22.801 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.801 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow primary affinity = true 2022-11-27T16:46:22.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd allow pg remap = true 2022-11-27T16:46:22.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:46:22.802 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:46:22.803 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on no sortbitwise = false 2022-11-27T16:46:22.803 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:46:22.804 INFO:teuthology.orchestra.run.smithi145.stdout: mon warn on too few osds = false 2022-11-27T16:46:22.804 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:46:22.804 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:46:22.805 INFO:teuthology.orchestra.run.smithi145.stdout: mon_allow_pool_size_one = true 2022-11-27T16:46:22.805 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.806 INFO:teuthology.orchestra.run.smithi145.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:46:22.806 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.807 INFO:teuthology.orchestra.run.smithi145.stdout: osd default data pool replay window = 5 2022-11-27T16:46:22.807 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.808 INFO:teuthology.orchestra.run.smithi145.stdout: mon allow pool delete = true 2022-11-27T16:46:22.808 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.809 INFO:teuthology.orchestra.run.smithi145.stdout: mon cluster log file level = debug 2022-11-27T16:46:22.809 INFO:teuthology.orchestra.run.smithi145.stdout: debug asserts on shutdown = true 2022-11-27T16:46:22.810 INFO:teuthology.orchestra.run.smithi145.stdout: mon health detail to clog = false 2022-11-27T16:46:22.810 INFO:teuthology.orchestra.run.smithi145.stdout: mon host = "172.21.15.145,172.21.15.201,[v2:172.21.15.201:3301,v1:172.21.15.201:6790]" 2022-11-27T16:46:22.811 INFO:teuthology.orchestra.run.smithi145.stdout: ms die on skipped message = False 2022-11-27T16:46:22.811 INFO:teuthology.orchestra.run.smithi145.stdout: fsid = 0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7 2022-11-27T16:46:22.811 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.812 INFO:teuthology.orchestra.run.smithi145.stdout:[osd] 2022-11-27T16:46:22.812 INFO:teuthology.orchestra.run.smithi145.stdout: osd journal size = 100 2022-11-27T16:46:22.813 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.813 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub load threshold = 5.0 2022-11-27T16:46:22.814 INFO:teuthology.orchestra.run.smithi145.stdout: osd scrub max interval = 600 2022-11-27T16:46:22.814 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.814 INFO:teuthology.orchestra.run.smithi145.stdout: osd recover clone overlap = true 2022-11-27T16:46:22.815 INFO:teuthology.orchestra.run.smithi145.stdout: osd recovery max chunk = 1048576 2022-11-27T16:46:22.815 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.815 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug shutdown = true 2022-11-27T16:46:22.816 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug op order = true 2022-11-27T16:46:22.816 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug verify stray on activate = true 2022-11-27T16:46:22.817 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.817 INFO:teuthology.orchestra.run.smithi145.stdout: osd open classes on start = true 2022-11-27T16:46:22.817 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug pg log writeout = true 2022-11-27T16:46:22.818 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.818 INFO:teuthology.orchestra.run.smithi145.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:46:22.818 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.818 INFO:teuthology.orchestra.run.smithi145.stdout: osd map max advance = 10 2022-11-27T16:46:22.819 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.819 INFO:teuthology.orchestra.run.smithi145.stdout: journal zero on create = true 2022-11-27T16:46:22.819 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.819 INFO:teuthology.orchestra.run.smithi145.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:46:22.820 INFO:teuthology.orchestra.run.smithi145.stdout: filestore apply finisher threads = 3 2022-11-27T16:46:22.820 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.820 INFO:teuthology.orchestra.run.smithi145.stdout: bdev debug aio = true 2022-11-27T16:46:22.820 INFO:teuthology.orchestra.run.smithi145.stdout: osd debug misdirected ops = true 2022-11-27T16:46:22.821 INFO:teuthology.orchestra.run.smithi145.stdout: bdev async discard = True 2022-11-27T16:46:22.821 INFO:teuthology.orchestra.run.smithi145.stdout: bdev enable discard = True 2022-11-27T16:46:22.821 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore allocator = bitmap 2022-11-27T16:46:22.821 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore block size = 96636764160 2022-11-27T16:46:22.822 INFO:teuthology.orchestra.run.smithi145.stdout: bluestore fsck on mount = True 2022-11-27T16:46:22.822 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluefs = 1/20 2022-11-27T16:46:22.822 INFO:teuthology.orchestra.run.smithi145.stdout: debug bluestore = 1/20 2022-11-27T16:46:22.822 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2022-11-27T16:46:22.822 INFO:teuthology.orchestra.run.smithi145.stdout: debug osd = 20 2022-11-27T16:46:22.823 INFO:teuthology.orchestra.run.smithi145.stdout: debug rocksdb = 4/10 2022-11-27T16:46:22.823 INFO:teuthology.orchestra.run.smithi145.stdout: memstore device bytes = 200000000 2022-11-27T16:46:22.823 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:46:22.823 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd full ratio = 0.9 2022-11-27T16:46:22.824 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:46:22.824 INFO:teuthology.orchestra.run.smithi145.stdout: osd failsafe full ratio = 1.0 2022-11-27T16:46:22.824 INFO:teuthology.orchestra.run.smithi145.stdout: osd mon report interval = 5 2022-11-27T16:46:22.824 INFO:teuthology.orchestra.run.smithi145.stdout: osd objectstore = memstore 2022-11-27T16:46:22.825 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2022-11-27T16:46:22.825 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.825 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr] 2022-11-27T16:46:22.826 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2022-11-27T16:46:22.826 INFO:teuthology.orchestra.run.smithi145.stdout: debug mgr = 20 2022-11-27T16:46:22.826 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2022-11-27T16:46:22.827 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2022-11-27T16:46:22.827 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:46:22.827 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:46:22.827 INFO:teuthology.orchestra.run.smithi145.stdout: mgr/telemetry/nag = false 2022-11-27T16:46:22.828 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.828 INFO:teuthology.orchestra.run.smithi145.stdout:[mon] 2022-11-27T16:46:22.828 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2022-11-27T16:46:22.829 INFO:teuthology.orchestra.run.smithi145.stdout: debug mon = 20 2022-11-27T16:46:22.829 INFO:teuthology.orchestra.run.smithi145.stdout: debug paxos = 20 2022-11-27T16:46:22.829 INFO:teuthology.orchestra.run.smithi145.stdout: debug auth = 20 2022-11-27T16:46:22.829 INFO:teuthology.orchestra.run.smithi145.stdout: mon data avail warn = 5 2022-11-27T16:46:22.830 INFO:teuthology.orchestra.run.smithi145.stdout: mon mgr mkfs grace = 240 2022-11-27T16:46:22.830 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:46:22.830 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd reporter subtree level = osd 2022-11-27T16:46:22.831 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd prime pg temp = true 2022-11-27T16:46:22.831 INFO:teuthology.orchestra.run.smithi145.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:46:22.831 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.832 INFO:teuthology.orchestra.run.smithi145.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:46:22.832 INFO:teuthology.orchestra.run.smithi145.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:46:22.832 INFO:teuthology.orchestra.run.smithi145.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:46:22.832 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.832 INFO:teuthology.orchestra.run.smithi145.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:46:22.833 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:46:22.833 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:46:22.833 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.833 INFO:teuthology.orchestra.run.smithi145.stdout: # 1m isn't quite enough 2022-11-27T16:46:22.834 INFO:teuthology.orchestra.run.smithi145.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:46:22.834 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.834 INFO:teuthology.orchestra.run.smithi145.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:46:22.834 INFO:teuthology.orchestra.run.smithi145.stdout: mon op complaint time = 120 2022-11-27T16:46:22.835 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd backfillfull ratio = 0.6 2022-11-27T16:46:22.835 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd full ratio = 0.7 2022-11-27T16:46:22.835 INFO:teuthology.orchestra.run.smithi145.stdout: mon osd nearfull ratio = 0.6 2022-11-27T16:46:22.835 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:22.835 INFO:teuthology.orchestra.run.smithi145.stdout:[client] 2022-11-27T16:46:22.836 INFO:teuthology.orchestra.run.smithi145.stdout: rgw cache enabled = true 2022-11-27T16:46:22.836 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable ops log = true 2022-11-27T16:46:22.836 INFO:teuthology.orchestra.run.smithi145.stdout: rgw enable usage log = true 2022-11-27T16:46:22.836 INFO:teuthology.orchestra.run.smithi145.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:46:22.837 INFO:teuthology.orchestra.run.smithi145.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:46:22.837 INFO:teuthology.orchestra.run.smithi145.stdout: client mount timeout = 600 2022-11-27T16:46:22.837 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2022-11-27T16:46:22.837 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2022-11-27T16:46:22.837 INFO:teuthology.orchestra.run.smithi145.stdout: pid file = "" 2022-11-27T16:46:22.838 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2022-11-27T16:46:22.838 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2022-11-27T16:46:22.838 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.a] 2022-11-27T16:46:22.839 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.b] 2022-11-27T16:46:22.839 INFO:teuthology.orchestra.run.smithi145.stdout:[mon.c] 2022-11-27T16:46:22.839 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2022-11-27T16:46:22.839 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2022-11-27T16:46:22.840 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2022-11-27T16:46:22.840 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2022-11-27T16:46:22.840 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2022-11-27T16:46:22.840 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2022-11-27T16:46:22.841 INFO:teuthology.orchestra.run.smithi145.stdout: debug objectcacher = 20 2022-11-27T16:46:22.841 INFO:teuthology.orchestra.run.smithi145.stdout: debug objecter = 20 2022-11-27T16:46:22.841 INFO:teuthology.orchestra.run.smithi145.stdout:[client.1] 2022-11-27T16:46:22.841 INFO:teuthology.orchestra.run.smithi145.stdout: debug client = 20 2022-11-27T16:46:22.842 INFO:teuthology.orchestra.run.smithi145.stdout: debug objectcacher = 20 2022-11-27T16:46:22.842 INFO:teuthology.orchestra.run.smithi145.stdout: debug objecter = 20 2022-11-27T16:46:22.842 INFO:teuthology.orchestra.run.smithi145.stdout:[mds] 2022-11-27T16:46:22.842 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds = 20 2022-11-27T16:46:22.843 INFO:teuthology.orchestra.run.smithi145.stdout: debug mds balancer = 20 2022-11-27T16:46:22.843 INFO:teuthology.orchestra.run.smithi145.stdout: debug ms = 1 2022-11-27T16:46:22.843 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug frag = True 2022-11-27T16:46:22.843 INFO:teuthology.orchestra.run.smithi145.stdout: mds debug scatterstat = True 2022-11-27T16:46:22.844 INFO:teuthology.orchestra.run.smithi145.stdout: mds op complaint time = 180 2022-11-27T16:46:22.844 INFO:teuthology.orchestra.run.smithi145.stdout: mds verify scatter = True 2022-11-27T16:46:22.844 INFO:teuthology.orchestra.run.smithi145.stdout: osd op complaint time = 180 2022-11-27T16:46:22.844 INFO:teuthology.orchestra.run.smithi145.stdout: rados mon op timeout = 900 2022-11-27T16:46:22.845 INFO:teuthology.orchestra.run.smithi145.stdout: rados osd op timeout = 900 2022-11-27T16:46:22.847 INFO:teuthology.orchestra.run.smithi201.stdout:[global] 2022-11-27T16:46:22.847 INFO:teuthology.orchestra.run.smithi201.stdout: chdir = "" 2022-11-27T16:46:22.847 INFO:teuthology.orchestra.run.smithi201.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2022-11-27T16:46:22.847 INFO:teuthology.orchestra.run.smithi201.stdout: auth supported = cephx 2022-11-27T16:46:22.848 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.848 INFO:teuthology.orchestra.run.smithi201.stdout: filestore xattr use omap = true 2022-11-27T16:46:22.848 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.848 INFO:teuthology.orchestra.run.smithi201.stdout: mon clock drift allowed = 1.000 2022-11-27T16:46:22.849 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.849 INFO:teuthology.orchestra.run.smithi201.stdout: osd crush chooseleaf type = 0 2022-11-27T16:46:22.849 INFO:teuthology.orchestra.run.smithi201.stdout: auth debug = true 2022-11-27T16:46:22.849 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.850 INFO:teuthology.orchestra.run.smithi201.stdout: ms die on old message = true 2022-11-27T16:46:22.850 INFO:teuthology.orchestra.run.smithi201.stdout: ms die on bug = true 2022-11-27T16:46:22.850 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.850 INFO:teuthology.orchestra.run.smithi201.stdout: mon max pg per osd = 10000 # >= luminous 2022-11-27T16:46:22.851 INFO:teuthology.orchestra.run.smithi201.stdout: mon pg warn max object skew = 0 2022-11-27T16:46:22.851 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.851 INFO:teuthology.orchestra.run.smithi201.stdout: # disable pg_autoscaler by default for new pools 2022-11-27T16:46:22.851 INFO:teuthology.orchestra.run.smithi201.stdout: osd_pool_default_pg_autoscale_mode = off 2022-11-27T16:46:22.852 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.852 INFO:teuthology.orchestra.run.smithi201.stdout: osd pool default size = 2 2022-11-27T16:46:22.852 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.853 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd allow primary affinity = true 2022-11-27T16:46:22.853 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd allow pg remap = true 2022-11-27T16:46:22.853 INFO:teuthology.orchestra.run.smithi201.stdout: mon warn on legacy crush tunables = false 2022-11-27T16:46:22.853 INFO:teuthology.orchestra.run.smithi201.stdout: mon warn on crush straw calc version zero = false 2022-11-27T16:46:22.854 INFO:teuthology.orchestra.run.smithi201.stdout: mon warn on no sortbitwise = false 2022-11-27T16:46:22.854 INFO:teuthology.orchestra.run.smithi201.stdout: mon warn on osd down out interval zero = false 2022-11-27T16:46:22.854 INFO:teuthology.orchestra.run.smithi201.stdout: mon warn on too few osds = false 2022-11-27T16:46:22.854 INFO:teuthology.orchestra.run.smithi201.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2022-11-27T16:46:22.855 INFO:teuthology.orchestra.run.smithi201.stdout: mon_warn_on_pool_no_redundancy = false 2022-11-27T16:46:22.855 INFO:teuthology.orchestra.run.smithi201.stdout: mon_allow_pool_size_one = true 2022-11-27T16:46:22.855 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.855 INFO:teuthology.orchestra.run.smithi201.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2022-11-27T16:46:22.856 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.856 INFO:teuthology.orchestra.run.smithi201.stdout: osd default data pool replay window = 5 2022-11-27T16:46:22.856 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.856 INFO:teuthology.orchestra.run.smithi201.stdout: mon allow pool delete = true 2022-11-27T16:46:22.856 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.857 INFO:teuthology.orchestra.run.smithi201.stdout: mon cluster log file level = debug 2022-11-27T16:46:22.857 INFO:teuthology.orchestra.run.smithi201.stdout: debug asserts on shutdown = true 2022-11-27T16:46:22.857 INFO:teuthology.orchestra.run.smithi201.stdout: mon health detail to clog = false 2022-11-27T16:46:22.857 INFO:teuthology.orchestra.run.smithi201.stdout: mon host = "172.21.15.145,172.21.15.201,[v2:172.21.15.201:3301,v1:172.21.15.201:6790]" 2022-11-27T16:46:22.857 INFO:teuthology.orchestra.run.smithi201.stdout: ms die on skipped message = False 2022-11-27T16:46:22.858 INFO:teuthology.orchestra.run.smithi201.stdout: fsid = 0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7 2022-11-27T16:46:22.858 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.858 INFO:teuthology.orchestra.run.smithi201.stdout:[osd] 2022-11-27T16:46:22.858 INFO:teuthology.orchestra.run.smithi201.stdout: osd journal size = 100 2022-11-27T16:46:22.859 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.859 INFO:teuthology.orchestra.run.smithi201.stdout: osd scrub load threshold = 5.0 2022-11-27T16:46:22.859 INFO:teuthology.orchestra.run.smithi201.stdout: osd scrub max interval = 600 2022-11-27T16:46:22.859 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.860 INFO:teuthology.orchestra.run.smithi201.stdout: osd recover clone overlap = true 2022-11-27T16:46:22.860 INFO:teuthology.orchestra.run.smithi201.stdout: osd recovery max chunk = 1048576 2022-11-27T16:46:22.860 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.860 INFO:teuthology.orchestra.run.smithi201.stdout: osd debug shutdown = true 2022-11-27T16:46:22.861 INFO:teuthology.orchestra.run.smithi201.stdout: osd debug op order = true 2022-11-27T16:46:22.861 INFO:teuthology.orchestra.run.smithi201.stdout: osd debug verify stray on activate = true 2022-11-27T16:46:22.861 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.861 INFO:teuthology.orchestra.run.smithi201.stdout: osd open classes on start = true 2022-11-27T16:46:22.861 INFO:teuthology.orchestra.run.smithi201.stdout: osd debug pg log writeout = true 2022-11-27T16:46:22.862 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.862 INFO:teuthology.orchestra.run.smithi201.stdout: osd deep scrub update digest min age = 30 2022-11-27T16:46:22.862 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.863 INFO:teuthology.orchestra.run.smithi201.stdout: osd map max advance = 10 2022-11-27T16:46:22.863 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.864 INFO:teuthology.orchestra.run.smithi201.stdout: journal zero on create = true 2022-11-27T16:46:22.864 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.864 INFO:teuthology.orchestra.run.smithi201.stdout: filestore ondisk finisher threads = 3 2022-11-27T16:46:22.864 INFO:teuthology.orchestra.run.smithi201.stdout: filestore apply finisher threads = 3 2022-11-27T16:46:22.864 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.865 INFO:teuthology.orchestra.run.smithi201.stdout: bdev debug aio = true 2022-11-27T16:46:22.865 INFO:teuthology.orchestra.run.smithi201.stdout: osd debug misdirected ops = true 2022-11-27T16:46:22.865 INFO:teuthology.orchestra.run.smithi201.stdout: bdev async discard = True 2022-11-27T16:46:22.865 INFO:teuthology.orchestra.run.smithi201.stdout: bdev enable discard = True 2022-11-27T16:46:22.865 INFO:teuthology.orchestra.run.smithi201.stdout: bluestore allocator = bitmap 2022-11-27T16:46:22.866 INFO:teuthology.orchestra.run.smithi201.stdout: bluestore block size = 96636764160 2022-11-27T16:46:22.866 INFO:teuthology.orchestra.run.smithi201.stdout: bluestore fsck on mount = True 2022-11-27T16:46:22.866 INFO:teuthology.orchestra.run.smithi201.stdout: debug bluefs = 1/20 2022-11-27T16:46:22.866 INFO:teuthology.orchestra.run.smithi201.stdout: debug bluestore = 1/20 2022-11-27T16:46:22.867 INFO:teuthology.orchestra.run.smithi201.stdout: debug ms = 1 2022-11-27T16:46:22.867 INFO:teuthology.orchestra.run.smithi201.stdout: debug osd = 20 2022-11-27T16:46:22.867 INFO:teuthology.orchestra.run.smithi201.stdout: debug rocksdb = 4/10 2022-11-27T16:46:22.868 INFO:teuthology.orchestra.run.smithi201.stdout: memstore device bytes = 200000000 2022-11-27T16:46:22.868 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd backfillfull_ratio = 0.85 2022-11-27T16:46:22.868 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd full ratio = 0.9 2022-11-27T16:46:22.868 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd nearfull ratio = 0.8 2022-11-27T16:46:22.868 INFO:teuthology.orchestra.run.smithi201.stdout: osd failsafe full ratio = 1.0 2022-11-27T16:46:22.869 INFO:teuthology.orchestra.run.smithi201.stdout: osd mon report interval = 5 2022-11-27T16:46:22.869 INFO:teuthology.orchestra.run.smithi201.stdout: osd objectstore = memstore 2022-11-27T16:46:22.869 INFO:teuthology.orchestra.run.smithi201.stdout: osd op complaint time = 180 2022-11-27T16:46:22.869 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.870 INFO:teuthology.orchestra.run.smithi201.stdout:[mgr] 2022-11-27T16:46:22.870 INFO:teuthology.orchestra.run.smithi201.stdout: debug ms = 1 2022-11-27T16:46:22.870 INFO:teuthology.orchestra.run.smithi201.stdout: debug mgr = 20 2022-11-27T16:46:22.870 INFO:teuthology.orchestra.run.smithi201.stdout: debug mon = 20 2022-11-27T16:46:22.871 INFO:teuthology.orchestra.run.smithi201.stdout: debug auth = 20 2022-11-27T16:46:22.871 INFO:teuthology.orchestra.run.smithi201.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:46:22.871 INFO:teuthology.orchestra.run.smithi201.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:46:22.871 INFO:teuthology.orchestra.run.smithi201.stdout: mgr/telemetry/nag = false 2022-11-27T16:46:22.872 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.872 INFO:teuthology.orchestra.run.smithi201.stdout:[mon] 2022-11-27T16:46:22.872 INFO:teuthology.orchestra.run.smithi201.stdout: debug ms = 1 2022-11-27T16:46:22.872 INFO:teuthology.orchestra.run.smithi201.stdout: debug mon = 20 2022-11-27T16:46:22.873 INFO:teuthology.orchestra.run.smithi201.stdout: debug paxos = 20 2022-11-27T16:46:22.873 INFO:teuthology.orchestra.run.smithi201.stdout: debug auth = 20 2022-11-27T16:46:22.873 INFO:teuthology.orchestra.run.smithi201.stdout: mon data avail warn = 5 2022-11-27T16:46:22.873 INFO:teuthology.orchestra.run.smithi201.stdout: mon mgr mkfs grace = 240 2022-11-27T16:46:22.874 INFO:teuthology.orchestra.run.smithi201.stdout: mon reweight min pgs per osd = 4 2022-11-27T16:46:22.874 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd reporter subtree level = osd 2022-11-27T16:46:22.874 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd prime pg temp = true 2022-11-27T16:46:22.874 INFO:teuthology.orchestra.run.smithi201.stdout: mon reweight min bytes per osd = 10 2022-11-27T16:46:22.875 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.875 INFO:teuthology.orchestra.run.smithi201.stdout: # rotate auth tickets quickly to exercise renewal paths 2022-11-27T16:46:22.875 INFO:teuthology.orchestra.run.smithi201.stdout: auth mon ticket ttl = 660 # 11m 2022-11-27T16:46:22.875 INFO:teuthology.orchestra.run.smithi201.stdout: auth service ticket ttl = 240 # 4m 2022-11-27T16:46:22.876 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.876 INFO:teuthology.orchestra.run.smithi201.stdout: # don't complain about insecure global_id in the test suite 2022-11-27T16:46:22.876 INFO:teuthology.orchestra.run.smithi201.stdout: mon_warn_on_insecure_global_id_reclaim = false 2022-11-27T16:46:22.876 INFO:teuthology.orchestra.run.smithi201.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2022-11-27T16:46:22.877 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.877 INFO:teuthology.orchestra.run.smithi201.stdout: # 1m isn't quite enough 2022-11-27T16:46:22.877 INFO:teuthology.orchestra.run.smithi201.stdout: mon_down_mkfs_grace = 2m 2022-11-27T16:46:22.877 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.877 INFO:teuthology.orchestra.run.smithi201.stdout: mon_warn_on_filestore_osds = false 2022-11-27T16:46:22.878 INFO:teuthology.orchestra.run.smithi201.stdout: mon op complaint time = 120 2022-11-27T16:46:22.878 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd backfillfull ratio = 0.6 2022-11-27T16:46:22.878 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd full ratio = 0.7 2022-11-27T16:46:22.878 INFO:teuthology.orchestra.run.smithi201.stdout: mon osd nearfull ratio = 0.6 2022-11-27T16:46:22.879 INFO:teuthology.orchestra.run.smithi201.stdout: 2022-11-27T16:46:22.879 INFO:teuthology.orchestra.run.smithi201.stdout:[client] 2022-11-27T16:46:22.879 INFO:teuthology.orchestra.run.smithi201.stdout: rgw cache enabled = true 2022-11-27T16:46:22.879 INFO:teuthology.orchestra.run.smithi201.stdout: rgw enable ops log = true 2022-11-27T16:46:22.879 INFO:teuthology.orchestra.run.smithi201.stdout: rgw enable usage log = true 2022-11-27T16:46:22.880 INFO:teuthology.orchestra.run.smithi201.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2022-11-27T16:46:22.880 INFO:teuthology.orchestra.run.smithi201.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2022-11-27T16:46:22.880 INFO:teuthology.orchestra.run.smithi201.stdout: client mount timeout = 600 2022-11-27T16:46:22.880 INFO:teuthology.orchestra.run.smithi201.stdout: debug client = 20 2022-11-27T16:46:22.880 INFO:teuthology.orchestra.run.smithi201.stdout: debug ms = 1 2022-11-27T16:46:22.881 INFO:teuthology.orchestra.run.smithi201.stdout: pid file = "" 2022-11-27T16:46:22.881 INFO:teuthology.orchestra.run.smithi201.stdout: rados mon op timeout = 900 2022-11-27T16:46:22.881 INFO:teuthology.orchestra.run.smithi201.stdout: rados osd op timeout = 900 2022-11-27T16:46:22.881 INFO:teuthology.orchestra.run.smithi201.stdout:[mon.a] 2022-11-27T16:46:22.881 INFO:teuthology.orchestra.run.smithi201.stdout:[mon.b] 2022-11-27T16:46:22.882 INFO:teuthology.orchestra.run.smithi201.stdout:[mon.c] 2022-11-27T16:46:22.882 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.a] 2022-11-27T16:46:22.882 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.c] 2022-11-27T16:46:22.882 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.b] 2022-11-27T16:46:22.883 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.d] 2022-11-27T16:46:22.883 INFO:teuthology.orchestra.run.smithi201.stdout:[client.0] 2022-11-27T16:46:22.883 INFO:teuthology.orchestra.run.smithi201.stdout: debug client = 20 2022-11-27T16:46:22.883 INFO:teuthology.orchestra.run.smithi201.stdout: debug objectcacher = 20 2022-11-27T16:46:22.883 INFO:teuthology.orchestra.run.smithi201.stdout: debug objecter = 20 2022-11-27T16:46:22.884 INFO:teuthology.orchestra.run.smithi201.stdout:[client.1] 2022-11-27T16:46:22.884 INFO:teuthology.orchestra.run.smithi201.stdout: debug client = 20 2022-11-27T16:46:22.884 INFO:teuthology.orchestra.run.smithi201.stdout: debug objectcacher = 20 2022-11-27T16:46:22.884 INFO:teuthology.orchestra.run.smithi201.stdout: debug objecter = 20 2022-11-27T16:46:22.884 INFO:teuthology.orchestra.run.smithi201.stdout:[mds] 2022-11-27T16:46:22.885 INFO:teuthology.orchestra.run.smithi201.stdout: debug mds = 20 2022-11-27T16:46:22.885 INFO:teuthology.orchestra.run.smithi201.stdout: debug mds balancer = 20 2022-11-27T16:46:22.885 INFO:teuthology.orchestra.run.smithi201.stdout: debug ms = 1 2022-11-27T16:46:22.885 INFO:teuthology.orchestra.run.smithi201.stdout: mds debug frag = True 2022-11-27T16:46:22.886 INFO:teuthology.orchestra.run.smithi201.stdout: mds debug scatterstat = True 2022-11-27T16:46:22.886 INFO:teuthology.orchestra.run.smithi201.stdout: mds op complaint time = 180 2022-11-27T16:46:22.886 INFO:teuthology.orchestra.run.smithi201.stdout: mds verify scatter = True 2022-11-27T16:46:22.887 INFO:teuthology.orchestra.run.smithi201.stdout: osd op complaint time = 180 2022-11-27T16:46:22.887 INFO:teuthology.orchestra.run.smithi201.stdout: rados mon op timeout = 900 2022-11-27T16:46:22.887 INFO:teuthology.orchestra.run.smithi201.stdout: rados osd op timeout = 900 2022-11-27T16:46:22.887 INFO:tasks.ceph:Creating admin key on mon.a... 2022-11-27T16:46:22.888 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2022-11-27T16:46:22.945 INFO:tasks.ceph:Copying monmap to all nodes... 2022-11-27T16:46:22.946 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:22.946 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2022-11-27T16:46:22.965 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:22.965 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2022-11-27T16:46:23.020 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi145.front.sepia.ceph.com 2022-11-27T16:46:23.021 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:23.021 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:46:23.022 DEBUG:teuthology.orchestra.run.smithi145:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:46:23.096 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:23.097 DEBUG:teuthology.orchestra.run.smithi145:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:23.153 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi201.front.sepia.ceph.com 2022-11-27T16:46:23.154 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:23.154 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd of=/etc/ceph/ceph.keyring 2022-11-27T16:46:23.155 DEBUG:teuthology.orchestra.run.smithi201:> sudo chmod 0644 /etc/ceph/ceph.keyring 2022-11-27T16:46:23.192 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:23.193 DEBUG:teuthology.orchestra.run.smithi201:> dd of=/home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:23.249 INFO:tasks.ceph:Setting up mon nodes... 2022-11-27T16:46:23.250 INFO:tasks.ceph:Setting up mgr nodes... 2022-11-27T16:46:23.250 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:46:23.303 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2022-11-27T16:46:23.306 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:46:23.368 INFO:teuthology.orchestra.run.smithi201.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2022-11-27T16:46:23.371 INFO:tasks.ceph:Setting up mds nodes... 2022-11-27T16:46:23.372 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:46:23.426 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2022-11-27T16:46:23.428 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2022-11-27T16:46:23.455 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:46:23.548 INFO:teuthology.orchestra.run.smithi145.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2022-11-27T16:46:23.551 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2022-11-27T16:46:23.578 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:46:23.631 INFO:teuthology.orchestra.run.smithi201.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2022-11-27T16:46:23.633 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2022-11-27T16:46:23.659 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:46:23.753 INFO:teuthology.orchestra.run.smithi201.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2022-11-27T16:46:23.756 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2022-11-27T16:46:23.783 INFO:tasks.ceph_client:Setting up client nodes... 2022-11-27T16:46:23.783 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2022-11-27T16:46:23.826 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.0.keyring 2022-11-27T16:46:23.839 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2022-11-27T16:46:23.922 INFO:teuthology.orchestra.run.smithi145.stdout:creating /etc/ceph/ceph.client.2.keyring 2022-11-27T16:46:23.935 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2022-11-27T16:46:23.978 INFO:teuthology.orchestra.run.smithi201.stdout:creating /etc/ceph/ceph.client.1.keyring 2022-11-27T16:46:23.991 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2022-11-27T16:46:24.074 INFO:teuthology.orchestra.run.smithi201.stdout:creating /etc/ceph/ceph.client.3.keyring 2022-11-27T16:46:24.087 INFO:tasks.ceph:Running mkfs on osd nodes... 2022-11-27T16:46:24.087 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi145.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi201.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2022-11-27T16:46:24.088 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2022-11-27T16:46:24.114 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:24.115 INFO:tasks.ceph:role: osd.0 2022-11-27T16:46:24.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com 2022-11-27T16:46:24.116 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:46:24.188 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:24.188 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:24.189 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:24.189 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:24.191 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:24.191 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:24.191 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:24.191 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:24.192 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:24.192 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:24.614 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2022-11-27T16:46:24.617 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2022-11-27T16:46:24.617 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2022-11-27T16:46:24.765 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2022-11-27T16:46:24.801 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2022-11-27T16:46:24.869 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:24.869 INFO:tasks.ceph:role: osd.1 2022-11-27T16:46:24.869 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com 2022-11-27T16:46:24.870 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:46:24.943 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:24.943 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:24.944 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:24.944 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:24.944 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:24.945 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:24.945 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:24.946 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:24.946 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:24.946 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:25.367 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2022-11-27T16:46:25.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2022-11-27T16:46:25.374 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2022-11-27T16:46:25.484 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2022-11-27T16:46:25.516 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2022-11-27T16:46:25.584 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:25.584 INFO:tasks.ceph:role: osd.2 2022-11-27T16:46:25.584 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com 2022-11-27T16:46:25.585 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:46:25.659 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:25.659 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:25.660 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:25.660 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:25.661 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:25.661 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:25.662 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:25.662 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:25.663 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:25.663 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:26.106 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2022-11-27T16:46:26.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2022-11-27T16:46:26.108 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2022-11-27T16:46:26.249 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2022-11-27T16:46:26.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2022-11-27T16:46:26.352 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:26.352 INFO:tasks.ceph:role: osd.3 2022-11-27T16:46:26.353 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com 2022-11-27T16:46:26.353 DEBUG:teuthology.orchestra.run.smithi145:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:46:26.427 INFO:teuthology.orchestra.run.smithi145.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:26.428 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:26.428 INFO:teuthology.orchestra.run.smithi145.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:26.428 INFO:teuthology.orchestra.run.smithi145.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:26.428 INFO:teuthology.orchestra.run.smithi145.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:26.429 INFO:teuthology.orchestra.run.smithi145.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:26.429 INFO:teuthology.orchestra.run.smithi145.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:26.429 INFO:teuthology.orchestra.run.smithi145.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:26.429 INFO:teuthology.orchestra.run.smithi145.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:26.430 INFO:teuthology.orchestra.run.smithi145.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:26.861 INFO:teuthology.orchestra.run.smithi145.stdout:Discarding blocks...Done. 2022-11-27T16:46:26.863 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi145.front.sepia.ceph.com -o noatime 2022-11-27T16:46:26.864 DEBUG:teuthology.orchestra.run.smithi145:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2022-11-27T16:46:27.020 DEBUG:teuthology.orchestra.run.smithi145:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2022-11-27T16:46:27.057 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:27.151 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.148+0000 7fba9fd8e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2022-11-27T16:46:27.151 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.148+0000 7fba9fd8e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2022-11-27T16:46:27.152 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.149+0000 7fba9fd8e5c0 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2022-11-27T16:46:27.158 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2022-11-27T16:46:27.188 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:27.275 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.272+0000 7f418f87b5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2022-11-27T16:46:27.276 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.272+0000 7f418f87b5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2022-11-27T16:46:27.276 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.272+0000 7f418f87b5c0 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2022-11-27T16:46:27.282 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2022-11-27T16:46:27.309 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:27.393 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.390+0000 7f351b5e85c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2022-11-27T16:46:27.393 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.390+0000 7f351b5e85c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2022-11-27T16:46:27.394 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.391+0000 7f351b5e85c0 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2022-11-27T16:46:27.401 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2022-11-27T16:46:27.468 DEBUG:teuthology.orchestra.run.smithi145:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:27.554 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.551+0000 7f638df9a5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2022-11-27T16:46:27.554 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.551+0000 7f638df9a5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2022-11-27T16:46:27.555 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:27.552+0000 7f638df9a5c0 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2022-11-27T16:46:27.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2022-11-27T16:46:27.587 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2022-11-27T16:46:27.620 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:27.621 INFO:tasks.ceph:role: osd.4 2022-11-27T16:46:27.621 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi201.front.sepia.ceph.com 2022-11-27T16:46:27.622 DEBUG:teuthology.orchestra.run.smithi201:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2022-11-27T16:46:27.698 INFO:teuthology.orchestra.run.smithi201.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:27.699 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:27.699 INFO:teuthology.orchestra.run.smithi201.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:27.700 INFO:teuthology.orchestra.run.smithi201.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:27.700 INFO:teuthology.orchestra.run.smithi201.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:27.700 INFO:teuthology.orchestra.run.smithi201.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:27.700 INFO:teuthology.orchestra.run.smithi201.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:27.701 INFO:teuthology.orchestra.run.smithi201.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:27.701 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:27.701 INFO:teuthology.orchestra.run.smithi201.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:28.131 INFO:teuthology.orchestra.run.smithi201.stdout:Discarding blocks...Done. 2022-11-27T16:46:28.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi201.front.sepia.ceph.com -o noatime 2022-11-27T16:46:28.134 DEBUG:teuthology.orchestra.run.smithi201:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2022-11-27T16:46:28.325 DEBUG:teuthology.orchestra.run.smithi201:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2022-11-27T16:46:28.361 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2022-11-27T16:46:28.430 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:28.430 INFO:tasks.ceph:role: osd.5 2022-11-27T16:46:28.431 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi201.front.sepia.ceph.com 2022-11-27T16:46:28.431 DEBUG:teuthology.orchestra.run.smithi201:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2022-11-27T16:46:28.506 INFO:teuthology.orchestra.run.smithi201.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:28.506 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:28.506 INFO:teuthology.orchestra.run.smithi201.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:28.507 INFO:teuthology.orchestra.run.smithi201.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:28.507 INFO:teuthology.orchestra.run.smithi201.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:28.507 INFO:teuthology.orchestra.run.smithi201.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:28.507 INFO:teuthology.orchestra.run.smithi201.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:28.508 INFO:teuthology.orchestra.run.smithi201.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:28.508 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:28.508 INFO:teuthology.orchestra.run.smithi201.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:28.919 INFO:teuthology.orchestra.run.smithi201.stdout:Discarding blocks...Done. 2022-11-27T16:46:28.920 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi201.front.sepia.ceph.com -o noatime 2022-11-27T16:46:28.920 DEBUG:teuthology.orchestra.run.smithi201:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2022-11-27T16:46:29.039 DEBUG:teuthology.orchestra.run.smithi201:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2022-11-27T16:46:29.074 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2022-11-27T16:46:29.143 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:29.143 INFO:tasks.ceph:role: osd.6 2022-11-27T16:46:29.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi201.front.sepia.ceph.com 2022-11-27T16:46:29.144 DEBUG:teuthology.orchestra.run.smithi201:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2022-11-27T16:46:29.219 INFO:teuthology.orchestra.run.smithi201.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:29.219 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:29.220 INFO:teuthology.orchestra.run.smithi201.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:29.220 INFO:teuthology.orchestra.run.smithi201.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:29.220 INFO:teuthology.orchestra.run.smithi201.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:29.220 INFO:teuthology.orchestra.run.smithi201.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:29.221 INFO:teuthology.orchestra.run.smithi201.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:29.221 INFO:teuthology.orchestra.run.smithi201.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:29.221 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:29.221 INFO:teuthology.orchestra.run.smithi201.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:29.662 INFO:teuthology.orchestra.run.smithi201.stdout:Discarding blocks...Done. 2022-11-27T16:46:29.664 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi201.front.sepia.ceph.com -o noatime 2022-11-27T16:46:29.665 DEBUG:teuthology.orchestra.run.smithi201:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2022-11-27T16:46:29.813 DEBUG:teuthology.orchestra.run.smithi201:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2022-11-27T16:46:29.848 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2022-11-27T16:46:29.918 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2022-11-27T16:46:29.919 INFO:tasks.ceph:role: osd.7 2022-11-27T16:46:29.920 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi201.front.sepia.ceph.com 2022-11-27T16:46:29.920 DEBUG:teuthology.orchestra.run.smithi201:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2022-11-27T16:46:29.994 INFO:teuthology.orchestra.run.smithi201.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2022-11-27T16:46:29.995 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 attr=2, projid32bit=1 2022-11-27T16:46:29.995 INFO:teuthology.orchestra.run.smithi201.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2022-11-27T16:46:29.996 INFO:teuthology.orchestra.run.smithi201.stdout: = reflink=1 bigtime=0 inobtcount=0 2022-11-27T16:46:29.996 INFO:teuthology.orchestra.run.smithi201.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2022-11-27T16:46:29.997 INFO:teuthology.orchestra.run.smithi201.stdout: = sunit=0 swidth=0 blks 2022-11-27T16:46:29.997 INFO:teuthology.orchestra.run.smithi201.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2022-11-27T16:46:29.998 INFO:teuthology.orchestra.run.smithi201.stdout:log =internal log bsize=4096 blocks=11443, version=2 2022-11-27T16:46:29.998 INFO:teuthology.orchestra.run.smithi201.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2022-11-27T16:46:29.999 INFO:teuthology.orchestra.run.smithi201.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2022-11-27T16:46:30.440 INFO:teuthology.orchestra.run.smithi201.stdout:Discarding blocks...Done. 2022-11-27T16:46:30.442 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi201.front.sepia.ceph.com -o noatime 2022-11-27T16:46:30.442 DEBUG:teuthology.orchestra.run.smithi201:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2022-11-27T16:46:30.616 DEBUG:teuthology.orchestra.run.smithi201:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2022-11-27T16:46:30.654 DEBUG:teuthology.orchestra.run.smithi201:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:30.744 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.742+0000 7fe9a1de95c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2022-11-27T16:46:30.744 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.742+0000 7fe9a1de95c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2022-11-27T16:46:30.745 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.742+0000 7fe9a1de95c0 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2022-11-27T16:46:30.750 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2022-11-27T16:46:30.777 DEBUG:teuthology.orchestra.run.smithi201:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:30.863 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.862+0000 7f9d92b495c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2022-11-27T16:46:30.864 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.862+0000 7f9d92b495c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2022-11-27T16:46:30.864 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.862+0000 7f9d92b495c0 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2022-11-27T16:46:30.869 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2022-11-27T16:46:30.896 DEBUG:teuthology.orchestra.run.smithi201:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:30.982 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.980+0000 7f599335c5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2022-11-27T16:46:30.983 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.981+0000 7f599335c5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2022-11-27T16:46:30.983 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:30.981+0000 7f599335c5c0 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2022-11-27T16:46:30.991 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2022-11-27T16:46:31.019 DEBUG:teuthology.orchestra.run.smithi201:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:31.105 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:31.103+0000 7f5eb60ea5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2022-11-27T16:46:31.106 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:31.104+0000 7f5eb60ea5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2022-11-27T16:46:31.106 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:31.104+0000 7f5eb60ea5c0 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2022-11-27T16:46:31.112 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2022-11-27T16:46:31.179 INFO:tasks.ceph:Reading keys from all nodes... 2022-11-27T16:46:31.180 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:31.180 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2022-11-27T16:46:31.213 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:31.213 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2022-11-27T16:46:31.283 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:31.284 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2022-11-27T16:46:31.353 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:31.353 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2022-11-27T16:46:31.422 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:31.422 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2022-11-27T16:46:31.492 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:31.492 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2022-11-27T16:46:31.560 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:31.561 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2022-11-27T16:46:31.628 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:31.628 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2022-11-27T16:46:31.655 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:31.655 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2022-11-27T16:46:31.721 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:31.722 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2022-11-27T16:46:31.787 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:31.788 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2022-11-27T16:46:31.853 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:31.854 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2022-11-27T16:46:31.920 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:31.921 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2022-11-27T16:46:31.986 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:31.987 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2022-11-27T16:46:32.053 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:32.054 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2022-11-27T16:46:32.074 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:32.075 DEBUG:teuthology.orchestra.run.smithi145:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2022-11-27T16:46:32.133 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:32.134 DEBUG:teuthology.orchestra.run.smithi201:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2022-11-27T16:46:32.152 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:32.152 DEBUG:teuthology.orchestra.run.smithi201:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2022-11-27T16:46:32.210 INFO:tasks.ceph:Adding keys to all mons... 2022-11-27T16:46:32.211 DEBUG:teuthology.orchestra.run.smithi145:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:46:32.214 DEBUG:teuthology.orchestra.run.smithi201:> sudo tee -a /etc/ceph/ceph.keyring 2022-11-27T16:46:32.254 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.y] 2022-11-27T16:46:32.254 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINj0ooMEhAA6/mb7IRz2c+pCBuGkzRITg== 2022-11-27T16:46:32.255 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.a] 2022-11-27T16:46:32.255 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINjAV9YGRAA9516se8kuP4SPP94IR/z8Q== 2022-11-27T16:46:32.255 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.c] 2022-11-27T16:46:32.255 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINjRxmnIBAAgnZNDie26MX0ZerMALuGDg== 2022-11-27T16:46:32.256 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.0] 2022-11-27T16:46:32.256 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBjlINj+I3xCBAAwwGmvTOQXTMKklyYBVmooQ== 2022-11-27T16:46:32.257 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.1] 2022-11-27T16:46:32.257 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBjlINjTk1QEBAAtldDu6MuPpVwUGAt1bt+vw== 2022-11-27T16:46:32.257 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.2] 2022-11-27T16:46:32.258 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBjlINjCc5hFxAAgT0mkSZOl60jJCk1CQnbcg== 2022-11-27T16:46:32.258 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.3] 2022-11-27T16:46:32.258 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBjlINjRSv5IBAAO3H5CiTgATq7Dc8KdzoEfg== 2022-11-27T16:46:32.259 INFO:teuthology.orchestra.run.smithi145.stdout:[mgr.x] 2022-11-27T16:46:32.259 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINjF0DlFRAAh2mIxK9uNWPbYbgd5V5DvQ== 2022-11-27T16:46:32.259 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.b] 2022-11-27T16:46:32.259 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINjugCWJRAA69aCTtKAOD7kLbIzxcNE9Q== 2022-11-27T16:46:32.260 INFO:teuthology.orchestra.run.smithi145.stdout:[mds.d] 2022-11-27T16:46:32.260 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINjm7PVLBAAjNysWvNc3YcWDKKiMdnNhA== 2022-11-27T16:46:32.260 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.4] 2022-11-27T16:46:32.260 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBmlINj7ShILBAAAGKviOVehMSYoSLgcg4/Iw== 2022-11-27T16:46:32.261 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.5] 2022-11-27T16:46:32.261 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBmlINj0WJrMxAA3C1DD2PutXNlbnJtl+9QuA== 2022-11-27T16:46:32.261 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.6] 2022-11-27T16:46:32.262 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBmlINjlmCCOhAA70Y1VdwwhvbGiNXo0FA2ZA== 2022-11-27T16:46:32.263 INFO:teuthology.orchestra.run.smithi145.stdout:[osd.7] 2022-11-27T16:46:32.263 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBnlINjo9Y6BhAAOfOnq4vkQ3Sq4WBTpfT8XQ== 2022-11-27T16:46:32.264 INFO:teuthology.orchestra.run.smithi145.stdout:[client.0] 2022-11-27T16:46:32.264 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINj8fE2MRAAoGLtcwIRub4y3EsARKWvCw== 2022-11-27T16:46:32.265 INFO:teuthology.orchestra.run.smithi145.stdout:[client.2] 2022-11-27T16:46:32.265 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINjKtfsNhAA9kOUXSaFqwqIvD/soL3S9A== 2022-11-27T16:46:32.265 INFO:teuthology.orchestra.run.smithi145.stdout:[client.1] 2022-11-27T16:46:32.266 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBflINjvptIOhAA9BbNxEGelMUWvklhGxXygg== 2022-11-27T16:46:32.266 INFO:teuthology.orchestra.run.smithi145.stdout:[client.3] 2022-11-27T16:46:32.266 INFO:teuthology.orchestra.run.smithi145.stdout: key = AQBglINj4a9iBBAA6lf2FR0NUK2nzqz82rzfGw== 2022-11-27T16:46:32.275 INFO:teuthology.orchestra.run.smithi201.stdout:[mgr.y] 2022-11-27T16:46:32.275 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINj0ooMEhAA6/mb7IRz2c+pCBuGkzRITg== 2022-11-27T16:46:32.276 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.a] 2022-11-27T16:46:32.276 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINjAV9YGRAA9516se8kuP4SPP94IR/z8Q== 2022-11-27T16:46:32.276 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.c] 2022-11-27T16:46:32.277 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINjRxmnIBAAgnZNDie26MX0ZerMALuGDg== 2022-11-27T16:46:32.278 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.0] 2022-11-27T16:46:32.278 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBjlINj+I3xCBAAwwGmvTOQXTMKklyYBVmooQ== 2022-11-27T16:46:32.278 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.1] 2022-11-27T16:46:32.278 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBjlINjTk1QEBAAtldDu6MuPpVwUGAt1bt+vw== 2022-11-27T16:46:32.279 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.2] 2022-11-27T16:46:32.279 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBjlINjCc5hFxAAgT0mkSZOl60jJCk1CQnbcg== 2022-11-27T16:46:32.279 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.3] 2022-11-27T16:46:32.279 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBjlINjRSv5IBAAO3H5CiTgATq7Dc8KdzoEfg== 2022-11-27T16:46:32.280 INFO:teuthology.orchestra.run.smithi201.stdout:[mgr.x] 2022-11-27T16:46:32.280 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINjF0DlFRAAh2mIxK9uNWPbYbgd5V5DvQ== 2022-11-27T16:46:32.280 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.b] 2022-11-27T16:46:32.281 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINjugCWJRAA69aCTtKAOD7kLbIzxcNE9Q== 2022-11-27T16:46:32.281 INFO:teuthology.orchestra.run.smithi201.stdout:[mds.d] 2022-11-27T16:46:32.281 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINjm7PVLBAAjNysWvNc3YcWDKKiMdnNhA== 2022-11-27T16:46:32.281 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.4] 2022-11-27T16:46:32.281 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBmlINj7ShILBAAAGKviOVehMSYoSLgcg4/Iw== 2022-11-27T16:46:32.282 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.5] 2022-11-27T16:46:32.282 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBmlINj0WJrMxAA3C1DD2PutXNlbnJtl+9QuA== 2022-11-27T16:46:32.282 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.6] 2022-11-27T16:46:32.283 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBmlINjlmCCOhAA70Y1VdwwhvbGiNXo0FA2ZA== 2022-11-27T16:46:32.284 INFO:teuthology.orchestra.run.smithi201.stdout:[osd.7] 2022-11-27T16:46:32.284 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBnlINjo9Y6BhAAOfOnq4vkQ3Sq4WBTpfT8XQ== 2022-11-27T16:46:32.284 INFO:teuthology.orchestra.run.smithi201.stdout:[client.0] 2022-11-27T16:46:32.285 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINj8fE2MRAAoGLtcwIRub4y3EsARKWvCw== 2022-11-27T16:46:32.285 INFO:teuthology.orchestra.run.smithi201.stdout:[client.2] 2022-11-27T16:46:32.286 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINjKtfsNhAA9kOUXSaFqwqIvD/soL3S9A== 2022-11-27T16:46:32.286 INFO:teuthology.orchestra.run.smithi201.stdout:[client.1] 2022-11-27T16:46:32.287 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBflINjvptIOhAA9BbNxEGelMUWvklhGxXygg== 2022-11-27T16:46:32.287 INFO:teuthology.orchestra.run.smithi201.stdout:[client.3] 2022-11-27T16:46:32.287 INFO:teuthology.orchestra.run.smithi201.stdout: key = AQBglINj4a9iBBAA6lf2FR0NUK2nzqz82rzfGw== 2022-11-27T16:46:32.289 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:46:32.312 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:46:32.383 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.386 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.445 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.448 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.504 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.508 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.560 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.563 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.654 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.657 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.710 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.712 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.764 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:46:32.767 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2022-11-27T16:46:32.821 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.824 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.879 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.882 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2022-11-27T16:46:32.940 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.943 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.996 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:32.998 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:33.052 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:33.054 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:33.107 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:33.110 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2022-11-27T16:46:33.164 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.166 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.222 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.224 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.280 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.284 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.341 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.344 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2022-11-27T16:46:33.479 INFO:tasks.ceph:Running mkfs on mon nodes... 2022-11-27T16:46:33.480 DEBUG:teuthology.orchestra.run.smithi145:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2022-11-27T16:46:33.507 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:46:33.605 DEBUG:teuthology.orchestra.run.smithi145:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2022-11-27T16:46:33.632 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2022-11-27T16:46:33.660 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:46:33.757 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2022-11-27T16:46:33.785 DEBUG:teuthology.orchestra.run.smithi201:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2022-11-27T16:46:33.852 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2022-11-27T16:46:33.952 DEBUG:teuthology.orchestra.run.smithi201:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2022-11-27T16:46:33.979 DEBUG:teuthology.orchestra.run.smithi145:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:33.982 DEBUG:teuthology.orchestra.run.smithi201:> rm -- /home/ubuntu/cephtest/ceph.monmap 2022-11-27T16:46:34.036 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2022-11-27T16:46:34.037 INFO:tasks.ceph.mon.a:Restarting daemon 2022-11-27T16:46:34.038 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2022-11-27T16:46:34.042 INFO:tasks.ceph.mon.a:Started 2022-11-27T16:46:34.042 INFO:tasks.ceph.mon.b:Restarting daemon 2022-11-27T16:46:34.043 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2022-11-27T16:46:34.079 INFO:tasks.ceph.mon.b:Started 2022-11-27T16:46:34.079 INFO:tasks.ceph.mon.c:Restarting daemon 2022-11-27T16:46:34.080 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2022-11-27T16:46:34.082 INFO:tasks.ceph.mon.c:Started 2022-11-27T16:46:34.082 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2022-11-27T16:46:34.083 INFO:tasks.ceph.mgr.y:Restarting daemon 2022-11-27T16:46:34.083 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2022-11-27T16:46:34.086 INFO:tasks.ceph.mgr.y:Started 2022-11-27T16:46:34.086 INFO:tasks.ceph.mgr.x:Restarting daemon 2022-11-27T16:46:34.087 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2022-11-27T16:46:34.089 INFO:tasks.ceph.mgr.x:Started 2022-11-27T16:46:34.090 INFO:tasks.ceph:Setting crush tunables to default 2022-11-27T16:46:34.090 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd crush tunables default 2022-11-27T16:46:34.223 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.217+0000 7fc8893f9700 1 Processor -- start 2022-11-27T16:46:34.224 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.217+0000 7fc8893f9700 1 -- start start 2022-11-27T16:46:34.225 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8893f9700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.225 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8893f9700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.226 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8893f9700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88406a500 con 0x7fc8840b1a50 2022-11-27T16:46:34.226 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8893f9700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8840cdd90 con 0x7fc8840cad50 2022-11-27T16:46:34.227 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8893f9700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8840cc2d0 con 0x7fc8840cead0 2022-11-27T16:46:34.227 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8827fc700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.229 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc882ffd700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fc8840b1a50 0x7fc8840cbbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49200/0 (socket says 172.21.15.145:49200) 2022-11-27T16:46:34.230 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc882ffd700 1 -- 172.21.15.145:0/3411146376 learned_addr learned my addr 172.21.15.145:0/3411146376 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:34.231 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8827fc700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 msgr2=0x7fc8840ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:46:34.231 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8827fc700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 msgr2=0x7fc8840ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:46:34.232 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8827fc700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:46:34.232 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8827fc700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:46:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8837fe700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.233 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8837fe700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 msgr2=0x7fc88406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-27T16:46:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8837fe700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 msgr2=0x7fc88406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:46:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8837fe700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:46:34.234 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.218+0000 7fc8837fe700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2022-11-27T16:46:34.422 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8827fc700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.423 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8837fe700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.423 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8827fc700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 msgr2=0x7fc8840ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2022-11-27T16:46:34.423 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8827fc700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 msgr2=0x7fc8840ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:46:34.424 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8827fc700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:46:34.424 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8827fc700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-11-27T16:46:34.424 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8837fe700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 msgr2=0x7fc88406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2022-11-27T16:46:34.424 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8837fe700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 msgr2=0x7fc88406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2022-11-27T16:46:34.425 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8837fe700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2022-11-27T16:46:34.425 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.419+0000 7fc8837fe700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2022-11-27T16:46:34.426 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.423+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1918065236 0 0) 0x7fc88406a500 con 0x7fc8840b1a50 2022-11-27T16:46:34.426 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.423+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8700035b0 con 0x7fc8840b1a50 2022-11-27T16:46:34.429 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4239782294 0 0) 0x7fc8700035b0 con 0x7fc8840b1a50 2022-11-27T16:46:34.429 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 msgr2=0x7fc88406a990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:46:34.429 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.201:3300/0 conn(0x7fc8840cead0 0x7fc88406a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.429 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 msgr2=0x7fc8840ce590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2022-11-27T16:46:34.430 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 --2- 172.21.15.145:0/3411146376 >> v2:172.21.15.145:3300/0 conn(0x7fc8840cad50 0x7fc8840ce590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.430 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840cef10 con 0x7fc8840b1a50 2022-11-27T16:46:34.430 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1749869138 0 0) 0x7fc874001670 con 0x7fc8840b1a50 2022-11-27T16:46:34.431 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 >> v1:172.21.15.145:6789/0 conn(0x7fc8840b1a50 legacy=0x7fc8840cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:34.431 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc870003a00 0x7fc870003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.431 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870004850 0x7fc870004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc870005200 0x7fc87000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840cdd90 con 0x7fc870003a00 2022-11-27T16:46:34.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840cc2d0 con 0x7fc870004850 2022-11-27T16:46:34.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87000c0e0 con 0x7fc870005200 2022-11-27T16:46:34.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc882ffd700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc870003a00 0x7fc870003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc8837fe700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc870005200 0x7fc87000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.424+0000 7fc8827fc700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870004850 0x7fc870004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.425+0000 7fc882ffd700 1 -- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc870005200 msgr2=0x7fc87000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:34.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.425+0000 7fc882ffd700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc870005200 0x7fc87000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.425+0000 7fc882ffd700 1 -- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870004850 msgr2=0x7fc870004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:34.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.425+0000 7fc882ffd700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870004850 0x7fc870004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.425+0000 7fc882ffd700 1 -- 172.21.15.145:0/3411146376 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8700035b0 con 0x7fc870003a00 2022-11-27T16:46:34.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.425+0000 7fc8837fe700 1 -- 172.21.15.145:0/3411146376 reap_dead start 2022-11-27T16:46:34.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.425+0000 7fc882ffd700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc870003a00 0x7fc870003e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c2f0 tx=0x7fc870005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b351dd7239fe5fba server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:34.435 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.426+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874007c40 con 0x7fc870003a00 2022-11-27T16:46:34.435 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.426+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87400cdb0 con 0x7fc870003a00 2022-11-27T16:46:34.435 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.426+0000 7fc881ffb700 1 -- 172.21.15.145:0/3411146376 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874005ba0 con 0x7fc870003a00 2022-11-27T16:46:34.438 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.427+0000 7fc8893f9700 1 -- 172.21.15.145:0/3411146376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc870003a00 msgr2=0x7fc870003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.427+0000 7fc8893f9700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc870003a00 0x7fc870003e10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc87400c2f0 tx=0x7fc870005880 comp rx=0 tx=0).stop 2022-11-27T16:46:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.427+0000 7fc8893f9700 1 -- 172.21.15.145:0/3411146376 shutdown_connections 2022-11-27T16:46:34.439 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.427+0000 7fc8893f9700 1 --2- 172.21.15.145:0/3411146376 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc870003a00 0x7fc870003e10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.427+0000 7fc8893f9700 1 -- 172.21.15.145:0/3411146376 >> 172.21.15.145:0/3411146376 conn(0x7fc8840999c0 msgr2=0x7fc8840d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.427+0000 7fc8893f9700 1 -- 172.21.15.145:0/3411146376 shutdown_connections 2022-11-27T16:46:34.440 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.427+0000 7fc8893f9700 1 -- 172.21.15.145:0/3411146376 wait complete. 2022-11-27T16:46:34.441 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.429+0000 7fc8893f9700 1 Processor -- start 2022-11-27T16:46:34.441 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.429+0000 7fc8893f9700 1 -- start start 2022-11-27T16:46:34.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8893f9700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870003a00 0x7fc8840b5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8893f9700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88415b3a0 0x7fc8841587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8893f9700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc88415b810 0x7fc884158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:34.443 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8893f9700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88414c0b0 con 0x7fc88415b3a0 2022-11-27T16:46:34.443 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8893f9700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840b5190 con 0x7fc870003a00 2022-11-27T16:46:34.444 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8893f9700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840b42c0 con 0x7fc88415b810 2022-11-27T16:46:34.444 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc882ffd700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870003a00 0x7fc8840b5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.445 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88415b3a0 0x7fc8841587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.445 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc882ffd700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870003a00 0x7fc8840b5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.145:60892/0 (socket says 172.21.15.145:60892) 2022-11-27T16:46:34.445 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88415b3a0 0x7fc8841587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54362/0 (socket says 172.21.15.145:54362) 2022-11-27T16:46:34.446 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc882ffd700 1 -- 172.21.15.145:0/2582698503 learned_addr learned my addr 172.21.15.145:0/2582698503 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:34.446 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8837fe700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc88415b810 0x7fc884158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:34.447 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 -- 172.21.15.145:0/2582698503 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc88415b810 msgr2=0x7fc884158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:34.447 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc88415b810 0x7fc884158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.448 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 -- 172.21.15.145:0/2582698503 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870003a00 msgr2=0x7fc8840b5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:34.448 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870003a00 0x7fc8840b5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.449 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 -- 172.21.15.145:0/2582698503 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8840b4580 con 0x7fc88415b3a0 2022-11-27T16:46:34.449 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.430+0000 7fc8827fc700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88415b3a0 0x7fc8841587a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8840b31d0 tx=0x7fc87800ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:34.449 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.431+0000 7fc86b7fe700 1 -- 172.21.15.145:0/2582698503 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87801e410 con 0x7fc88415b3a0 2022-11-27T16:46:34.450 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.431+0000 7fc86b7fe700 1 -- 172.21.15.145:0/2582698503 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87800c040 con 0x7fc88415b3a0 2022-11-27T16:46:34.450 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.431+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8840b4860 con 0x7fc88415b3a0 2022-11-27T16:46:34.450 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.431+0000 7fc86b7fe700 1 -- 172.21.15.145:0/2582698503 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87801e770 con 0x7fc88415b3a0 2022-11-27T16:46:34.451 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.431+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8840b4e20 con 0x7fc88415b3a0 2022-11-27T16:46:34.451 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.431+0000 7fc86b7fe700 1 -- 172.21.15.145:0/2582698503 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc878025070 con 0x7fc88415b3a0 2022-11-27T16:46:34.452 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.431+0000 7fc86b7fe700 1 -- 172.21.15.145:0/2582698503 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc8780158f0 con 0x7fc88415b3a0 2022-11-27T16:46:34.452 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.432+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8840b4fe0 con 0x7fc88415b3a0 2022-11-27T16:46:34.452 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.434+0000 7fc86b7fe700 1 -- 172.21.15.145:0/2582698503 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7fc87803d6d0 con 0x7fc88415b3a0 2022-11-27T16:46:34.512 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.509+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fc8840b5300 con 0x7fc88415b3a0 2022-11-27T16:46:34.515 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.512+0000 7fc86b7fe700 1 -- 172.21.15.145:0/2582698503 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc87802c070 con 0x7fc88415b3a0 2022-11-27T16:46:34.517 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.513+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88415b3a0 msgr2=0x7fc8841587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:34.517 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.513+0000 7fc8893f9700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88415b3a0 0x7fc8841587a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc8840b31d0 tx=0x7fc87800ea70 comp rx=0 tx=0).stop 2022-11-27T16:46:34.518 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.513+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 shutdown_connections 2022-11-27T16:46:34.518 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.513+0000 7fc8893f9700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc88415b810 0x7fc884158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.518 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.513+0000 7fc8893f9700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc870003a00 0x7fc8840b5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.518 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.513+0000 7fc8893f9700 1 --2- 172.21.15.145:0/2582698503 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc88415b3a0 0x7fc8841587a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:34.519 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.513+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 >> 172.21.15.145:0/2582698503 conn(0x7fc8840999c0 msgr2=0x7fc88406d910 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:34.519 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.514+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 shutdown_connections 2022-11-27T16:46:34.519 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:34.514+0000 7fc8893f9700 1 -- 172.21.15.145:0/2582698503 wait complete. 2022-11-27T16:46:34.519 INFO:teuthology.orchestra.run.smithi145.stderr:adjusted tunables profile to default 2022-11-27T16:46:34.529 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2022-11-27T16:46:34.530 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:34.530 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2022-11-27T16:46:34.541 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:34.539+0000 7f8e2cc53200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:46:34.544 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:34.542+0000 7f4c500ad200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2022-11-27T16:46:34.557 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:34.558 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2022-11-27T16:46:34.603 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:34.602+0000 7f8e2cc53200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:46:34.623 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:34.623 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2022-11-27T16:46:34.672 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:34.670+0000 7f4c500ad200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2022-11-27T16:46:34.678 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:34.676+0000 7f8e2cc53200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:46:34.690 DEBUG:teuthology.orchestra.run.smithi145:> set -ex 2022-11-27T16:46:34.691 DEBUG:teuthology.orchestra.run.smithi145:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2022-11-27T16:46:34.750 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:34.747+0000 7f4c500ad200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2022-11-27T16:46:34.756 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:34.754+0000 7f8e2cc53200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:46:34.757 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:34.757 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2022-11-27T16:46:34.783 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:34.784 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2022-11-27T16:46:34.830 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:34.827+0000 7f4c500ad200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2022-11-27T16:46:34.850 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:34.851 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2022-11-27T16:46:34.904 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:34.902+0000 7f8e2cc53200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:46:34.918 DEBUG:teuthology.orchestra.run.smithi201:> set -ex 2022-11-27T16:46:34.918 DEBUG:teuthology.orchestra.run.smithi201:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2022-11-27T16:46:34.969 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:34.966+0000 7f4c500ad200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2022-11-27T16:46:34.984 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2022-11-27T16:46:35.151 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.141+0000 7f8f025bb700 1 Processor -- start 2022-11-27T16:46:35.152 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.141+0000 7f8f025bb700 1 -- start start 2022-11-27T16:46:35.152 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f025bb700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f8efc069380 0x7f8efc0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.152 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f025bb700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc070af0 0x7f8efc1605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.152 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f025bb700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8efc071380 con 0x7f8efc0706f0 2022-11-27T16:46:35.153 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f025bb700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8efc071940 con 0x7f8efc069380 2022-11-27T16:46:35.153 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f025bb700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc1601b0 con 0x7f8efc070af0 2022-11-27T16:46:35.153 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f00b58700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc070af0 0x7f8efc1605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.154 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f00b58700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc070af0 0x7f8efc1605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38854/0 (socket says 172.21.15.201:38854) 2022-11-27T16:46:35.155 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8f00b58700 1 -- 172.21.15.201:0/2537716073 learned_addr learned my addr 172.21.15.201:0/2537716073 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:35.155 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.142+0000 7f8efbfff700 1 --2- 172.21.15.201:0/2537716073 >> v2:172.21.15.145:3300/0 conn(0x7f8efc069380 0x7f8efc0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.156 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8f00b58700 1 -- 172.21.15.201:0/2537716073 >> v2:172.21.15.145:3300/0 conn(0x7f8efc069380 msgr2=0x7f8efc0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:35.156 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8f00b58700 1 --2- 172.21.15.201:0/2537716073 >> v2:172.21.15.145:3300/0 conn(0x7f8efc069380 0x7f8efc0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.157 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8f00b58700 1 -- 172.21.15.201:0/2537716073 >> v1:172.21.15.145:6789/0 conn(0x7f8efc0706f0 legacy=0x7f8efc069750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.157 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8f00b58700 1 -- 172.21.15.201:0/2537716073 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8efc071660 con 0x7f8efc070af0 2022-11-27T16:46:35.158 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8efbfff700 1 --2- 172.21.15.201:0/2537716073 >> v2:172.21.15.145:3300/0 conn(0x7f8efc069380 0x7f8efc0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:35.158 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8f00b58700 1 --2- 172.21.15.201:0/2537716073 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc070af0 0x7f8efc1605d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e50 tx=0x7f8ef400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a400b3e1c44fa9e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.158 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8efaffd700 1 -- 172.21.15.201:0/2537716073 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400e070 con 0x7f8efc070af0 2022-11-27T16:46:35.159 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8efaffd700 1 -- 172.21.15.201:0/2537716073 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef400b9c0 con 0x7f8efc070af0 2022-11-27T16:46:35.159 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8efaffd700 1 -- 172.21.15.201:0/2537716073 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400bd70 con 0x7f8efc070af0 2022-11-27T16:46:35.160 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8f025bb700 1 -- 172.21.15.201:0/2537716073 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc070af0 msgr2=0x7f8efc1605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.160 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.143+0000 7f8f025bb700 1 --2- 172.21.15.201:0/2537716073 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc070af0 0x7f8efc1605d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009e50 tx=0x7f8ef400b330 comp rx=0 tx=0).stop 2022-11-27T16:46:35.160 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.144+0000 7f8f025bb700 1 -- 172.21.15.201:0/2537716073 shutdown_connections 2022-11-27T16:46:35.161 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.144+0000 7f8f025bb700 1 --2- 172.21.15.201:0/2537716073 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc070af0 0x7f8efc1605d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.161 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.144+0000 7f8f025bb700 1 --2- 172.21.15.201:0/2537716073 >> v2:172.21.15.145:3300/0 conn(0x7f8efc069380 0x7f8efc0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.161 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.144+0000 7f8f025bb700 1 -- 172.21.15.201:0/2537716073 >> 172.21.15.201:0/2537716073 conn(0x7f8efc06a490 msgr2=0x7f8efc06a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.161 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.144+0000 7f8f025bb700 1 -- 172.21.15.201:0/2537716073 shutdown_connections 2022-11-27T16:46:35.162 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.144+0000 7f8f025bb700 1 -- 172.21.15.201:0/2537716073 wait complete. 2022-11-27T16:46:35.162 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.145+0000 7f8f025bb700 1 Processor -- start 2022-11-27T16:46:35.162 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.145+0000 7f8f025bb700 1 -- start start 2022-11-27T16:46:35.162 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f025bb700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8efc069380 0x7f8efc1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.162 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f025bb700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8efc070af0 0x7f8efc15ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.163 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f025bb700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc15f4d0 0x7f8efc0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.163 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f025bb700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc1f5d60 con 0x7f8efc069380 2022-11-27T16:46:35.163 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f025bb700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc15f950 con 0x7f8efc070af0 2022-11-27T16:46:35.163 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f025bb700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8efc1f6180 con 0x7f8efc15f4d0 2022-11-27T16:46:35.164 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8efc070af0 0x7f8efc15ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.164 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc15f4d0 0x7f8efc0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.164 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc15f4d0 0x7f8efc0c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38860/0 (socket says 172.21.15.201:38860) 2022-11-27T16:46:35.164 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8efb7fe700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8efc070af0 0x7f8efc15ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56160/0 (socket says 172.21.15.201:56160) 2022-11-27T16:46:35.164 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 -- 172.21.15.201:0/3531750818 learned_addr learned my addr 172.21.15.201:0/3531750818 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:35.165 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 -- 172.21.15.201:0/3531750818 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8efc070af0 msgr2=0x7f8efc15ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.165 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 --2- 172.21.15.201:0/3531750818 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8efc070af0 0x7f8efc15ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.165 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 -- 172.21.15.201:0/3531750818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8efc069380 msgr2=0x7f8efc1f4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.165 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 --2- 172.21.15.201:0/3531750818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8efc069380 0x7f8efc1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.166 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 -- 172.21.15.201:0/3531750818 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4009b30 con 0x7f8efc15f4d0 2022-11-27T16:46:35.166 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f00b58700 1 --2- 172.21.15.201:0/3531750818 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc15f4d0 0x7f8efc0c4b10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4012000 tx=0x7f8ef4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.166 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8ef8ff9700 1 -- 172.21.15.201:0/3531750818 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef40048b0 con 0x7f8efc15f4d0 2022-11-27T16:46:35.166 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8ef8ff9700 1 -- 172.21.15.201:0/3531750818 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef40099b0 con 0x7f8efc15f4d0 2022-11-27T16:46:35.166 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8efc0c5870 con 0x7f8efc15f4d0 2022-11-27T16:46:35.167 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.146+0000 7f8ef8ff9700 1 -- 172.21.15.201:0/3531750818 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef401c680 con 0x7f8efc15f4d0 2022-11-27T16:46:35.167 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.147+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8efc1f56a0 con 0x7f8efc15f4d0 2022-11-27T16:46:35.167 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.147+0000 7f8ef8ff9700 1 -- 172.21.15.201:0/3531750818 <== mon.2 v2:172.21.15.201:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8ef4022030 con 0x7f8efc15f4d0 2022-11-27T16:46:35.167 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.147+0000 7f8ef8ff9700 1 -- 172.21.15.201:0/3531750818 <== mon.2 v2:172.21.15.201:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f8ef4025790 con 0x7f8efc15f4d0 2022-11-27T16:46:35.168 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.147+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8efc0739b0 con 0x7f8efc15f4d0 2022-11-27T16:46:35.168 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.150+0000 7f8ef8ff9700 1 -- 172.21.15.201:0/3531750818 <== mon.2 v2:172.21.15.201:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f8ef40436d0 con 0x7f8efc15f4d0 2022-11-27T16:46:35.231 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.229+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7f8efc0cc7b0 con 0x7f8efc15f4d0 2022-11-27T16:46:35.234 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.233+0000 7f8ef8ff9700 1 -- 172.21.15.201:0/3531750818 <== mon.2 v2:172.21.15.201:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8ef4032030 con 0x7f8efc15f4d0 2022-11-27T16:46:35.235 INFO:teuthology.orchestra.run.smithi201.stdout:0 2022-11-27T16:46:35.236 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc15f4d0 msgr2=0x7f8efc0c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.237 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 --2- 172.21.15.201:0/3531750818 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc15f4d0 0x7f8efc0c4b10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4012000 tx=0x7f8ef4004ad0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.237 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 shutdown_connections 2022-11-27T16:46:35.237 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 --2- 172.21.15.201:0/3531750818 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8efc15f4d0 0x7f8efc0c4b10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.237 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 --2- 172.21.15.201:0/3531750818 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8efc069380 0x7f8efc1f4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.238 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 --2- 172.21.15.201:0/3531750818 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8efc070af0 0x7f8efc15ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.238 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 >> 172.21.15.201:0/3531750818 conn(0x7f8efc06a490 msgr2=0x7f8efc15b430 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.238 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 shutdown_connections 2022-11-27T16:46:35.238 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.234+0000 7f8f025bb700 1 -- 172.21.15.201:0/3531750818 wait complete. 2022-11-27T16:46:35.247 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2022-11-27T16:46:35.365 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.363+0000 7f0b6f26d700 1 Processor -- start 2022-11-27T16:46:35.365 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.363+0000 7f0b6f26d700 1 -- start start 2022-11-27T16:46:35.368 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.364+0000 7f0b6f26d700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f0b680cbc30 0x7f0b68159db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.368 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.364+0000 7f0b6f26d700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6815a2f0 con 0x7f0b680ae7e0 2022-11-27T16:46:35.369 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.364+0000 7f0b6f26d700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6815d7b0 con 0x7f0b680708a0 2022-11-27T16:46:35.369 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.364+0000 7f0b6f26d700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6815c8c0 con 0x7f0b680cbc30 2022-11-27T16:46:35.369 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.364+0000 7f0b6d80a700 1 --1- >> v1:172.21.15.201:6789/0 conn(0x7f0b680708a0 0x7f0b68070ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.201:6789/0 says I am v1:172.21.15.201:56470/0 (socket says 172.21.15.201:56470) 2022-11-27T16:46:35.369 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.364+0000 7f0b6d80a700 1 -- 172.21.15.201:0/3737365197 learned_addr learned my addr 172.21.15.201:0/3737365197 (peer_addr_for_me v1:172.21.15.201:0/0) 2022-11-27T16:46:35.370 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b6d009700 1 --2- 172.21.15.201:0/3737365197 >> v2:172.21.15.145:3300/0 conn(0x7f0b680cbc30 0x7f0b68159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.370 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 <== mon.1 v1:172.21.15.201:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61981530 0 0) 0x7f0b6815d7b0 con 0x7f0b680708a0 2022-11-27T16:46:35.371 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 --> v1:172.21.15.201:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b500035b0 con 0x7f0b680708a0 2022-11-27T16:46:35.371 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754306842 0 0) 0x7f0b64001110 con 0x7f0b680ae7e0 2022-11-27T16:46:35.371 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b6815d7b0 con 0x7f0b680ae7e0 2022-11-27T16:46:35.371 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b6d009700 1 -- 172.21.15.201:0/3737365197 >> v1:172.21.15.201:6789/0 conn(0x7f0b680708a0 legacy=0x7f0b68070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.372 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 <== mon.1 v1:172.21.15.201:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087353404 0 0) 0x7f0b500035b0 con 0x7f0b680708a0 2022-11-27T16:46:35.372 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b6d009700 1 -- 172.21.15.201:0/3737365197 >> v1:172.21.15.145:6789/0 conn(0x7f0b680ae7e0 legacy=0x7f0b680cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.372 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b6d009700 1 -- 172.21.15.201:0/3737365197 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6815cba0 con 0x7f0b680cbc30 2022-11-27T16:46:35.372 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b6d009700 1 --2- 172.21.15.201:0/3737365197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 0x7f0b68159db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b580097a0 tx=0x7f0b58009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced7a1a2a20e4521 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.372 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60001a30 con 0x7f0b680cbc30 2022-11-27T16:46:35.373 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b58005c30 con 0x7f0b680cbc30 2022-11-27T16:46:35.373 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.365+0000 7f0b57fff700 1 -- 172.21.15.201:0/3737365197 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58004030 con 0x7f0b680cbc30 2022-11-27T16:46:35.373 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.366+0000 7f0b6f26d700 1 -- 172.21.15.201:0/3737365197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 msgr2=0x7f0b68159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.373 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.366+0000 7f0b6f26d700 1 --2- 172.21.15.201:0/3737365197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 0x7f0b68159db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b580097a0 tx=0x7f0b58009ab0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.373 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.366+0000 7f0b6f26d700 1 -- 172.21.15.201:0/3737365197 shutdown_connections 2022-11-27T16:46:35.374 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.366+0000 7f0b6f26d700 1 --2- 172.21.15.201:0/3737365197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 0x7f0b68159db0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.374 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.366+0000 7f0b6f26d700 1 -- 172.21.15.201:0/3737365197 >> 172.21.15.201:0/3737365197 conn(0x7f0b6806a490 msgr2=0x7f0b6806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.374 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.366+0000 7f0b6f26d700 1 -- 172.21.15.201:0/3737365197 shutdown_connections 2022-11-27T16:46:35.375 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.366+0000 7f0b6f26d700 1 -- 172.21.15.201:0/3737365197 wait complete. 2022-11-27T16:46:35.375 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.367+0000 7f0b6f26d700 1 Processor -- start 2022-11-27T16:46:35.375 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.367+0000 7f0b6f26d700 1 -- start start 2022-11-27T16:46:35.379 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6f26d700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f0b680708a0 0x7f0b680c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.380 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6f26d700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f0b680ae7e0 0x7f0b681510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.380 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6f26d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 0x7f0b680ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.381 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6f26d700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680c6b10 con 0x7f0b680cbc30 2022-11-27T16:46:35.381 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6f26d700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680c8e40 con 0x7f0b680ae7e0 2022-11-27T16:46:35.381 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6f26d700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680c93e0 con 0x7f0b680708a0 2022-11-27T16:46:35.381 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6c808700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f0b680ae7e0 0x7f0b681510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.381 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f0b680708a0 0x7f0b680c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.382 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f0b680708a0 0x7f0b680c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38868/0 (socket says 172.21.15.201:38868) 2022-11-27T16:46:35.382 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6c808700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f0b680ae7e0 0x7f0b681510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56172/0 (socket says 172.21.15.201:56172) 2022-11-27T16:46:35.382 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 -- 172.21.15.201:0/1085787062 learned_addr learned my addr 172.21.15.201:0/1085787062 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:35.382 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d80a700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 0x7f0b680ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.383 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 -- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f0b680ae7e0 msgr2=0x7f0b681510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.383 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f0b680ae7e0 0x7f0b681510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.383 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 -- 172.21.15.201:0/1085787062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 msgr2=0x7f0b680ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.384 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 0x7f0b680ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.384 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 -- 172.21.15.201:0/1085787062 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58009430 con 0x7f0b680708a0 2022-11-27T16:46:35.384 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6c808700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f0b680ae7e0 0x7f0b681510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:35.385 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6d009700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f0b680708a0 0x7f0b680c65d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009770 tx=0x7f0b58004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.385 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b55ffb700 1 -- 172.21.15.201:0/1085787062 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58003ab0 con 0x7f0b680708a0 2022-11-27T16:46:35.385 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b55ffb700 1 -- 172.21.15.201:0/1085787062 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b58003c10 con 0x7f0b680708a0 2022-11-27T16:46:35.385 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b680c9100 con 0x7f0b680708a0 2022-11-27T16:46:35.385 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.368+0000 7f0b55ffb700 1 -- 172.21.15.201:0/1085787062 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5801d430 con 0x7f0b680708a0 2022-11-27T16:46:35.386 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.369+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b68151610 con 0x7f0b680708a0 2022-11-27T16:46:35.386 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.369+0000 7f0b55ffb700 1 -- 172.21.15.201:0/1085787062 <== mon.2 v2:172.21.15.201:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f0b5802d070 con 0x7f0b680708a0 2022-11-27T16:46:35.386 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.369+0000 7f0b55ffb700 1 -- 172.21.15.201:0/1085787062 <== mon.2 v2:172.21.15.201:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f0b58026800 con 0x7f0b680708a0 2022-11-27T16:46:35.387 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.369+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b680abc70 con 0x7f0b680708a0 2022-11-27T16:46:35.387 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.372+0000 7f0b55ffb700 1 -- 172.21.15.201:0/1085787062 <== mon.2 v2:172.21.15.201:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f0b580456d0 con 0x7f0b680708a0 2022-11-27T16:46:35.455 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.453+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7f0b680739b0 con 0x7f0b680708a0 2022-11-27T16:46:35.458 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.457+0000 7f0b55ffb700 1 -- 172.21.15.201:0/1085787062 <== mon.2 v2:172.21.15.201:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0b58034030 con 0x7f0b680708a0 2022-11-27T16:46:35.459 INFO:teuthology.orchestra.run.smithi201.stdout:1 2022-11-27T16:46:35.460 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f0b680708a0 msgr2=0x7f0b680c65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.460 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f0b680708a0 0x7f0b680c65d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b58009770 tx=0x7f0b58004750 comp rx=0 tx=0).stop 2022-11-27T16:46:35.461 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 shutdown_connections 2022-11-27T16:46:35.461 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f0b680cbc30 0x7f0b680ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.461 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f0b680708a0 0x7f0b680c65d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.461 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 --2- 172.21.15.201:0/1085787062 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f0b680ae7e0 0x7f0b681510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.462 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 >> 172.21.15.201:0/1085787062 conn(0x7f0b6806a490 msgr2=0x7f0b68159900 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.462 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 shutdown_connections 2022-11-27T16:46:35.462 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.458+0000 7f0b6f26d700 1 -- 172.21.15.201:0/1085787062 wait complete. 2022-11-27T16:46:35.472 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2022-11-27T16:46:35.601 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.589+0000 7f321b2a4700 1 Processor -- start 2022-11-27T16:46:35.602 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.589+0000 7f321b2a4700 1 -- start start 2022-11-27T16:46:35.603 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f321b2a4700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f3214156120 0x7f32141564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.603 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f321b2a4700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f32140cd0d0 0x7f3214071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.603 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f321b2a4700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3214156a30 con 0x7f3214069380 2022-11-27T16:46:35.604 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f321b2a4700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321415d300 con 0x7f32140cd0d0 2022-11-27T16:46:35.604 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f321b2a4700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32140b1f90 con 0x7f3214156120 2022-11-27T16:46:35.604 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f3219040700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f3214156120 0x7f32141564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.605 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f3219040700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f3214156120 0x7f32141564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56176/0 (socket says 172.21.15.201:56176) 2022-11-27T16:46:35.605 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f3219040700 1 -- 172.21.15.201:0/2878141548 learned_addr learned my addr 172.21.15.201:0/2878141548 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:35.605 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f321883f700 1 --2- 172.21.15.201:0/2878141548 >> v2:172.21.15.145:3300/0 conn(0x7f32140cd0d0 0x7f3214071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.605 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f3219040700 1 -- 172.21.15.201:0/2878141548 >> v2:172.21.15.145:3300/0 conn(0x7f32140cd0d0 msgr2=0x7f3214071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:35.606 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f3219040700 1 --2- 172.21.15.201:0/2878141548 >> v2:172.21.15.145:3300/0 conn(0x7f32140cd0d0 0x7f3214071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.606 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f3219040700 1 -- 172.21.15.201:0/2878141548 >> v1:172.21.15.145:6789/0 conn(0x7f3214069380 legacy=0x7f3214069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.606 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.590+0000 7f3219040700 1 -- 172.21.15.201:0/2878141548 --> v2:172.21.15.201:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321415c680 con 0x7f3214156120 2022-11-27T16:46:35.606 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f3219040700 1 --2- 172.21.15.201:0/2878141548 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f32141564f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f321406f4f0 tx=0x7f3204009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977839684e23789f server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.607 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f3203fff700 1 -- 172.21.15.201:0/2878141548 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204005e30 con 0x7f3214156120 2022-11-27T16:46:35.607 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f3203fff700 1 -- 172.21.15.201:0/2878141548 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3204004030 con 0x7f3214156120 2022-11-27T16:46:35.607 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f3203fff700 1 -- 172.21.15.201:0/2878141548 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32040043e0 con 0x7f3214156120 2022-11-27T16:46:35.607 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 -- 172.21.15.201:0/2878141548 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 msgr2=0x7f32141564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.607 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 --2- 172.21.15.201:0/2878141548 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f32141564f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f321406f4f0 tx=0x7f3204009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.608 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 -- 172.21.15.201:0/2878141548 shutdown_connections 2022-11-27T16:46:35.608 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 --2- 172.21.15.201:0/2878141548 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f32141564f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.608 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 --2- 172.21.15.201:0/2878141548 >> v2:172.21.15.145:3300/0 conn(0x7f32140cd0d0 0x7f3214071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.608 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 -- 172.21.15.201:0/2878141548 >> 172.21.15.201:0/2878141548 conn(0x7f321406a490 msgr2=0x7f321406a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.609 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 -- 172.21.15.201:0/2878141548 shutdown_connections 2022-11-27T16:46:35.609 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.591+0000 7f321b2a4700 1 -- 172.21.15.201:0/2878141548 wait complete. 2022-11-27T16:46:35.609 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 Processor -- start 2022-11-27T16:46:35.609 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 -- start start 2022-11-27T16:46:35.609 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3214069380 0x7f32141f1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.610 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f32140cd0d0 0x7f32140c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.610 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f321415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.610 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32140ca4a0 con 0x7f3214069380 2022-11-27T16:46:35.611 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32140b2560 con 0x7f3214156120 2022-11-27T16:46:35.611 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321b2a4700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32140c9c00 con 0x7f32140cd0d0 2022-11-27T16:46:35.611 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f3219841700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f321415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.611 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.593+0000 7f321883f700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f32140cd0d0 0x7f32140c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.612 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f3219841700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f321415d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56184/0 (socket says 172.21.15.201:56184) 2022-11-27T16:46:35.612 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321883f700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f32140cd0d0 0x7f32140c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38880/0 (socket says 172.21.15.201:38880) 2022-11-27T16:46:35.612 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f3219841700 1 -- 172.21.15.201:0/2455429519 learned_addr learned my addr 172.21.15.201:0/2455429519 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:35.612 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f3219040700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3214069380 0x7f32141f1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.613 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321883f700 1 -- 172.21.15.201:0/2455429519 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 msgr2=0x7f321415d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.613 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321883f700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f321415d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.613 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321883f700 1 -- 172.21.15.201:0/2455429519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3214069380 msgr2=0x7f32141f1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.614 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321883f700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3214069380 0x7f32141f1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.614 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321883f700 1 -- 172.21.15.201:0/2455429519 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204009500 con 0x7f32140cd0d0 2022-11-27T16:46:35.614 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321883f700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f32140cd0d0 0x7f32140c7060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f321000e810 tx=0x7f321000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.614 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f3201ffb700 1 -- 172.21.15.201:0/2455429519 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321000c070 con 0x7f32140cd0d0 2022-11-27T16:46:35.615 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f3201ffb700 1 -- 172.21.15.201:0/2455429519 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3210004020 con 0x7f32140cd0d0 2022-11-27T16:46:35.615 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32140c9ee0 con 0x7f32140cd0d0 2022-11-27T16:46:35.615 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f3201ffb700 1 -- 172.21.15.201:0/2455429519 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32100043d0 con 0x7f32140cd0d0 2022-11-27T16:46:35.615 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f3201ffb700 1 -- 172.21.15.201:0/2455429519 <== mon.2 v2:172.21.15.201:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f32100036a0 con 0x7f32140cd0d0 2022-11-27T16:46:35.616 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.594+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32141f1650 con 0x7f32140cd0d0 2022-11-27T16:46:35.616 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.595+0000 7f3201ffb700 1 -- 172.21.15.201:0/2455429519 <== mon.2 v2:172.21.15.201:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f3210004530 con 0x7f32140cd0d0 2022-11-27T16:46:35.616 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.595+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32140b2330 con 0x7f32140cd0d0 2022-11-27T16:46:35.616 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.597+0000 7f3201ffb700 1 -- 172.21.15.201:0/2455429519 <== mon.2 v2:172.21.15.201:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f3210017030 con 0x7f32140cd0d0 2022-11-27T16:46:35.660 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:35.657+0000 7f4c500ad200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:46:35.680 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.678+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7f32140739b0 con 0x7f32140cd0d0 2022-11-27T16:46:35.683 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.682+0000 7f3201ffb700 1 -- 172.21.15.201:0/2455429519 <== mon.2 v2:172.21.15.201:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f321000f040 con 0x7f32140cd0d0 2022-11-27T16:46:35.684 INFO:teuthology.orchestra.run.smithi201.stdout:2 2022-11-27T16:46:35.685 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f32140cd0d0 msgr2=0x7f32140c7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.686 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f32140cd0d0 0x7f32140c7060 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f321000e810 tx=0x7f321000eb20 comp rx=0 tx=0).stop 2022-11-27T16:46:35.686 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 shutdown_connections 2022-11-27T16:46:35.686 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f3214156120 0x7f321415d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.686 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f3214069380 0x7f32141f1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.687 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 --2- 172.21.15.201:0/2455429519 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f32140cd0d0 0x7f32140c7060 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.687 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 >> 172.21.15.201:0/2455429519 conn(0x7f321406a490 msgr2=0x7f32140cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.687 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 shutdown_connections 2022-11-27T16:46:35.687 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.683+0000 7f321b2a4700 1 -- 172.21.15.201:0/2455429519 wait complete. 2022-11-27T16:46:35.697 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2022-11-27T16:46:35.725 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:35.724+0000 7f8e2cc53200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2022-11-27T16:46:35.737 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:35.733+0000 7f4c500ad200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:46:35.802 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:35.801+0000 7f8e2cc53200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2022-11-27T16:46:35.818 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.817+0000 7f51a3f39700 1 Processor -- start 2022-11-27T16:46:35.819 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.817+0000 7f51a3f39700 1 -- start start 2022-11-27T16:46:35.821 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a3f39700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f519c0c9350 0x7f519c15d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.822 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a3f39700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c06a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.822 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a3f39700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c069380 con 0x7f519c0c9720 2022-11-27T16:46:35.822 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a3f39700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c0695d0 con 0x7f519c0c9350 2022-11-27T16:46:35.823 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a3f39700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0698b0 con 0x7f519c0ab8e0 2022-11-27T16:46:35.823 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f519c0c9350 0x7f519c15d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.824 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a24d6700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c06a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.824 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f519c0c9350 0x7f519c15d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56186/0 (socket says 172.21.15.201:56186) 2022-11-27T16:46:35.824 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a24d6700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c06a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38886/0 (socket says 172.21.15.201:38886) 2022-11-27T16:46:35.824 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 -- 172.21.15.201:0/1042247183 learned_addr learned my addr 172.21.15.201:0/1042247183 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:35.825 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 -- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 msgr2=0x7f519c06a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:35.825 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 --2- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c06a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.825 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 -- 172.21.15.201:0/1042247183 >> v1:172.21.15.145:6789/0 conn(0x7f519c0c9720 legacy=0x7f519c0adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.825 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 -- 172.21.15.201:0/1042247183 --> v2:172.21.15.201:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c069b90 con 0x7f519c0c9350 2022-11-27T16:46:35.826 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a24d6700 1 --2- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c06a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:35.826 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a1cd5700 1 --2- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9350 0x7f519c15d0d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51980098f0 tx=0x7f5198009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee9461b4e4e7a42e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.827 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.818+0000 7f51a0cd3700 1 -- 172.21.15.201:0/1042247183 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198011830 con 0x7f519c0c9350 2022-11-27T16:46:35.828 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a0cd3700 1 -- 172.21.15.201:0/1042247183 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5198011990 con 0x7f519c0c9350 2022-11-27T16:46:35.828 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a0cd3700 1 -- 172.21.15.201:0/1042247183 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5198011d20 con 0x7f519c0c9350 2022-11-27T16:46:35.828 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 -- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9350 msgr2=0x7f519c15d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.828 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 --2- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9350 0x7f519c15d0d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51980098f0 tx=0x7f5198009c00 comp rx=0 tx=0).stop 2022-11-27T16:46:35.829 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 -- 172.21.15.201:0/1042247183 shutdown_connections 2022-11-27T16:46:35.829 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 --2- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c06a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.829 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 --2- 172.21.15.201:0/1042247183 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9350 0x7f519c15d0d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.829 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 -- 172.21.15.201:0/1042247183 >> 172.21.15.201:0/1042247183 conn(0x7f519c0cb780 msgr2=0x7f519c0cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.829 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 -- 172.21.15.201:0/1042247183 shutdown_connections 2022-11-27T16:46:35.830 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.819+0000 7f51a3f39700 1 -- 172.21.15.201:0/1042247183 wait complete. 2022-11-27T16:46:35.833 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.820+0000 7f51a3f39700 1 Processor -- start 2022-11-27T16:46:35.834 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a3f39700 1 -- start start 2022-11-27T16:46:35.834 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a3f39700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c0c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.834 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a3f39700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f519c0c9350 0x7f519c144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.834 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a3f39700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9720 0x7f519c0c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:35.835 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a3f39700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c145e20 con 0x7f519c0c9350 2022-11-27T16:46:35.835 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a3f39700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0c6270 con 0x7f519c0c9720 2022-11-27T16:46:35.835 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a3f39700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c145860 con 0x7f519c0ab8e0 2022-11-27T16:46:35.836 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a1cd5700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c0c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.836 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a24d6700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9720 0x7f519c0c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.836 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a24d6700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9720 0x7f519c0c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56194/0 (socket says 172.21.15.201:56194) 2022-11-27T16:46:35.836 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a1cd5700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c0c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38888/0 (socket says 172.21.15.201:38888) 2022-11-27T16:46:35.837 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a24d6700 1 -- 172.21.15.201:0/2685996942 learned_addr learned my addr 172.21.15.201:0/2685996942 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:35.837 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.821+0000 7f51a14d4700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f519c0c9350 0x7f519c144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:35.837 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a24d6700 1 -- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 msgr2=0x7f519c0c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.837 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a24d6700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c0c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.837 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a24d6700 1 -- 172.21.15.201:0/2685996942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f519c0c9350 msgr2=0x7f519c144ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.838 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a24d6700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f519c0c9350 0x7f519c144ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.838 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a24d6700 1 -- 172.21.15.201:0/2685996942 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009a70 con 0x7f519c0c9720 2022-11-27T16:46:35.838 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a1cd5700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:35.838 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a24d6700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9720 0x7f519c0c6700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f519400efc0 tx=0x7f519400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:35.839 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f518a7fc700 1 -- 172.21.15.201:0/2685996942 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519400ca90 con 0x7f519c0c9720 2022-11-27T16:46:35.839 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f518a7fc700 1 -- 172.21.15.201:0/2685996942 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519400f040 con 0x7f519c0c9720 2022-11-27T16:46:35.839 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5198009550 con 0x7f519c0c9720 2022-11-27T16:46:35.839 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f518a7fc700 1 -- 172.21.15.201:0/2685996942 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519400ce10 con 0x7f519c0c9720 2022-11-27T16:46:35.840 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c0c5f00 con 0x7f519c0c9720 2022-11-27T16:46:35.840 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f518a7fc700 1 -- 172.21.15.201:0/2685996942 <== mon.1 v2:172.21.15.201:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5194003a20 con 0x7f519c0c9720 2022-11-27T16:46:35.840 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.822+0000 7f518a7fc700 1 -- 172.21.15.201:0/2685996942 <== mon.1 v2:172.21.15.201:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f5194004750 con 0x7f519c0c9720 2022-11-27T16:46:35.840 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.823+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c0652c0 con 0x7f519c0c9720 2022-11-27T16:46:35.841 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.825+0000 7f518a7fc700 1 -- 172.21.15.201:0/2685996942 <== mon.1 v2:172.21.15.201:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f5194018030 con 0x7f519c0c9720 2022-11-27T16:46:35.859 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:35.857+0000 7f4c500ad200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:46:35.910 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.908+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f519c0c63e0 con 0x7f519c0c9720 2022-11-27T16:46:35.913 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.912+0000 7f518a7fc700 1 -- 172.21.15.201:0/2685996942 <== mon.1 v2:172.21.15.201:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5194003e20 con 0x7f519c0c9720 2022-11-27T16:46:35.914 INFO:teuthology.orchestra.run.smithi201.stdout:3 2022-11-27T16:46:35.915 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9720 msgr2=0x7f519c0c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:35.915 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9720 0x7f519c0c6700 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f519400efc0 tx=0x7f519400c3a0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.916 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 shutdown_connections 2022-11-27T16:46:35.916 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f519c0c9720 0x7f519c0c6700 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.916 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f519c0ab8e0 0x7f519c0c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.916 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 --2- 172.21.15.201:0/2685996942 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f519c0c9350 0x7f519c144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:35.917 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 >> 172.21.15.201:0/2685996942 conn(0x7f519c0cb780 msgr2=0x7f519c15c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:35.917 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 shutdown_connections 2022-11-27T16:46:35.917 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:35.913+0000 7f51a3f39700 1 -- 172.21.15.201:0/2685996942 wait complete. 2022-11-27T16:46:35.926 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2022-11-27T16:46:35.938 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:35.936+0000 7f4c500ad200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:46:35.939 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:35.938+0000 7f8e2cc53200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2022-11-27T16:46:36.026 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:36.024+0000 7f8e2cc53200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2022-11-27T16:46:36.061 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.059+0000 7f725d4ad700 1 Processor -- start 2022-11-27T16:46:36.061 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.059+0000 7f725d4ad700 1 -- start start 2022-11-27T16:46:36.068 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f725d4ad700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f72580b1a50 0x7f72580cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.068 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f725d4ad700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580ceca0 0x7f725806a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.068 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f725d4ad700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725806a500 con 0x7f72580cad50 2022-11-27T16:46:36.068 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f725d4ad700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72580cdd90 con 0x7f72580b1a50 2022-11-27T16:46:36.069 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f725d4ad700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72580cc4e0 con 0x7f72580ceca0 2022-11-27T16:46:36.069 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f72577fe700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580ceca0 0x7f725806a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.069 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f72577fe700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580ceca0 0x7f725806a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38898/0 (socket says 172.21.15.201:38898) 2022-11-27T16:46:36.069 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f72577fe700 1 -- 172.21.15.201:0/648354039 learned_addr learned my addr 172.21.15.201:0/648354039 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:36.069 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f7256ffd700 1 --2- 172.21.15.201:0/648354039 >> v2:172.21.15.201:3300/0 conn(0x7f72580b1a50 0x7f72580cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.070 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f72577fe700 1 -- 172.21.15.201:0/648354039 >> v2:172.21.15.201:3300/0 conn(0x7f72580b1a50 msgr2=0x7f72580cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:36.070 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f72577fe700 1 --2- 172.21.15.201:0/648354039 >> v2:172.21.15.201:3300/0 conn(0x7f72580b1a50 0x7f72580cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.070 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f72577fe700 1 -- 172.21.15.201:0/648354039 >> v1:172.21.15.201:6789/0 conn(0x7f72580cad50 legacy=0x7f72580ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.070 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.060+0000 7f72577fe700 1 -- 172.21.15.201:0/648354039 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72580cc7c0 con 0x7f72580ceca0 2022-11-27T16:46:36.071 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f7256ffd700 1 --2- 172.21.15.201:0/648354039 >> v2:172.21.15.201:3300/0 conn(0x7f72580b1a50 0x7f72580cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:36.071 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f72577fe700 1 --2- 172.21.15.201:0/648354039 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580ceca0 0x7f725806a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7250009e50 tx=0x7f725000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5c92689bae12dac server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.071 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f7255ffb700 1 -- 172.21.15.201:0/648354039 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725000e070 con 0x7f72580ceca0 2022-11-27T16:46:36.071 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f7255ffb700 1 -- 172.21.15.201:0/648354039 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725000b910 con 0x7f72580ceca0 2022-11-27T16:46:36.072 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f7255ffb700 1 -- 172.21.15.201:0/648354039 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725000bcc0 con 0x7f72580ceca0 2022-11-27T16:46:36.072 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 -- 172.21.15.201:0/648354039 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580ceca0 msgr2=0x7f725806a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.072 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 --2- 172.21.15.201:0/648354039 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580ceca0 0x7f725806a990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7250009e50 tx=0x7f725000b330 comp rx=0 tx=0).stop 2022-11-27T16:46:36.072 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 -- 172.21.15.201:0/648354039 shutdown_connections 2022-11-27T16:46:36.073 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 --2- 172.21.15.201:0/648354039 >> v2:172.21.15.201:3300/0 conn(0x7f72580b1a50 0x7f72580cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.073 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 --2- 172.21.15.201:0/648354039 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580ceca0 0x7f725806a990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.073 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 -- 172.21.15.201:0/648354039 >> 172.21.15.201:0/648354039 conn(0x7f72580999c0 msgr2=0x7f72580d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.073 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 -- 172.21.15.201:0/648354039 shutdown_connections 2022-11-27T16:46:36.074 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.061+0000 7f725d4ad700 1 -- 172.21.15.201:0/648354039 wait complete. 2022-11-27T16:46:36.074 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.063+0000 7f725d4ad700 1 Processor -- start 2022-11-27T16:46:36.074 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.063+0000 7f725d4ad700 1 -- start start 2022-11-27T16:46:36.074 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.063+0000 7f725d4ad700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72580b1a50 0x7f72581516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.074 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.063+0000 7f725d4ad700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580cad50 0x7f72580c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.075 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.063+0000 7f725d4ad700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f72580ceca0 0x7f72580c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.075 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f725d4ad700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72581f4a80 con 0x7f72580b1a50 2022-11-27T16:46:36.075 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f725d4ad700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72580b5a40 con 0x7f72580ceca0 2022-11-27T16:46:36.075 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f725d4ad700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7258151c30 con 0x7f72580cad50 2022-11-27T16:46:36.076 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580cad50 0x7f72580c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.076 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72577fe700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f72580ceca0 0x7f72580c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.076 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580cad50 0x7f72580c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38902/0 (socket says 172.21.15.201:38902) 2022-11-27T16:46:36.076 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 -- 172.21.15.201:0/3676872005 learned_addr learned my addr 172.21.15.201:0/3676872005 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:36.077 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72577fe700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f72580ceca0 0x7f72580c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56206/0 (socket says 172.21.15.201:56206) 2022-11-27T16:46:36.077 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f7256ffd700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72580b1a50 0x7f72581516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.077 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 -- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f72580ceca0 msgr2=0x7f72580c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.077 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f72580ceca0 0x7f72580c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.077 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 -- 172.21.15.201:0/3676872005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72580b1a50 msgr2=0x7f72581516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.078 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72580b1a50 0x7f72581516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.078 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 -- 172.21.15.201:0/3676872005 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72480094a0 con 0x7f72580cad50 2022-11-27T16:46:36.078 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f7256ffd700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72580b1a50 0x7f72581516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:36.078 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72577fe700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f72580ceca0 0x7f72580c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:36.079 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f72567fc700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580cad50 0x7f72580c9170 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f724000c4e0 tx=0x7f724000c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.079 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f724f7fe700 1 -- 172.21.15.201:0/3676872005 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400052d0 con 0x7f72580cad50 2022-11-27T16:46:36.079 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f724f7fe700 1 -- 172.21.15.201:0/3676872005 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7240005430 con 0x7f72580cad50 2022-11-27T16:46:36.079 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7250009b30 con 0x7f72580cad50 2022-11-27T16:46:36.080 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f724f7fe700 1 -- 172.21.15.201:0/3676872005 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72400038c0 con 0x7f72580cad50 2022-11-27T16:46:36.080 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72580b4dc0 con 0x7f72580cad50 2022-11-27T16:46:36.080 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.064+0000 7f724f7fe700 1 -- 172.21.15.201:0/3676872005 <== mon.2 v2:172.21.15.201:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f7240019030 con 0x7f72580cad50 2022-11-27T16:46:36.080 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.065+0000 7f724f7fe700 1 -- 172.21.15.201:0/3676872005 <== mon.2 v2:172.21.15.201:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f7240011d60 con 0x7f72580cad50 2022-11-27T16:46:36.080 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.065+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72580b5b80 con 0x7f72580cad50 2022-11-27T16:46:36.081 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.067+0000 7f724f7fe700 1 -- 172.21.15.201:0/3676872005 <== mon.2 v2:172.21.15.201:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f7240017070 con 0x7f72580cad50 2022-11-27T16:46:36.151 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.150+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f72580739b0 con 0x7f72580cad50 2022-11-27T16:46:36.155 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.153+0000 7f724f7fe700 1 -- 172.21.15.201:0/3676872005 <== mon.2 v2:172.21.15.201:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7240011860 con 0x7f72580cad50 2022-11-27T16:46:36.156 INFO:teuthology.orchestra.run.smithi201.stdout:4 2022-11-27T16:46:36.157 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580cad50 msgr2=0x7f72580c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.158 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580cad50 0x7f72580c9170 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f724000c4e0 tx=0x7f724000c7f0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.158 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 shutdown_connections 2022-11-27T16:46:36.159 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f72580ceca0 0x7f72580c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.159 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f72580b1a50 0x7f72581516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.160 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 --2- 172.21.15.201:0/3676872005 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f72580cad50 0x7f72580c9170 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.160 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 >> 172.21.15.201:0/3676872005 conn(0x7f72580999c0 msgr2=0x7f725806b680 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.160 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 shutdown_connections 2022-11-27T16:46:36.161 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.154+0000 7f725d4ad700 1 -- 172.21.15.201:0/3676872005 wait complete. 2022-11-27T16:46:36.168 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2022-11-27T16:46:36.290 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.288+0000 7f4056441700 1 Processor -- start 2022-11-27T16:46:36.290 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.288+0000 7f4056441700 1 -- start start 2022-11-27T16:46:36.293 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f4056441700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f4050156120 0x7f40501564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.294 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f4056441700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40500b1f90 con 0x7f4050069380 2022-11-27T16:46:36.294 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f4056441700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4050156a30 con 0x7f40500cd0d0 2022-11-27T16:46:36.294 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f4056441700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405015d300 con 0x7f4050156120 2022-11-27T16:46:36.294 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f404ffff700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f4050156120 0x7f40501564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.295 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f404ffff700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f4050156120 0x7f40501564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56208/0 (socket says 172.21.15.201:56208) 2022-11-27T16:46:36.295 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f404ffff700 1 -- 172.21.15.201:0/1014302495 learned_addr learned my addr 172.21.15.201:0/1014302495 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:36.295 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f404ffff700 1 -- 172.21.15.201:0/1014302495 >> v1:172.21.15.201:6789/0 conn(0x7f40500cd0d0 legacy=0x7f4050071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.295 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f404ffff700 1 -- 172.21.15.201:0/1014302495 >> v1:172.21.15.145:6789/0 conn(0x7f4050069380 legacy=0x7f4050069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.296 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.289+0000 7f404ffff700 1 -- 172.21.15.201:0/1014302495 --> v2:172.21.15.201:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405015c680 con 0x7f4050156120 2022-11-27T16:46:36.296 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f404ffff700 1 --2- 172.21.15.201:0/1014302495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050156120 0x7f40501564f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6821f333f9459c17 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.296 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f404effd700 1 -- 172.21.15.201:0/1014302495 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040005e30 con 0x7f4050156120 2022-11-27T16:46:36.296 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f404effd700 1 -- 172.21.15.201:0/1014302495 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040004030 con 0x7f4050156120 2022-11-27T16:46:36.297 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f404effd700 1 -- 172.21.15.201:0/1014302495 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400043e0 con 0x7f4050156120 2022-11-27T16:46:36.297 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f4056441700 1 -- 172.21.15.201:0/1014302495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050156120 msgr2=0x7f40501564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.297 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f4056441700 1 --2- 172.21.15.201:0/1014302495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050156120 0x7f40501564f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.297 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f4056441700 1 -- 172.21.15.201:0/1014302495 shutdown_connections 2022-11-27T16:46:36.298 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f4056441700 1 --2- 172.21.15.201:0/1014302495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050156120 0x7f40501564f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.298 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f4056441700 1 -- 172.21.15.201:0/1014302495 >> 172.21.15.201:0/1014302495 conn(0x7f405006a490 msgr2=0x7f405006a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.298 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.290+0000 7f4056441700 1 -- 172.21.15.201:0/1014302495 shutdown_connections 2022-11-27T16:46:36.298 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.291+0000 7f4056441700 1 -- 172.21.15.201:0/1014302495 wait complete. 2022-11-27T16:46:36.299 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.292+0000 7f4056441700 1 Processor -- start 2022-11-27T16:46:36.299 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.292+0000 7f4056441700 1 -- start start 2022-11-27T16:46:36.302 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.292+0000 7f4056441700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050069380 0x7f40500ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.302 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.292+0000 7f4056441700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f40500cd0d0 0x7f40500c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.303 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f4056441700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4050156120 0x7f4050155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.303 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f4056441700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c8630 con 0x7f4050156120 2022-11-27T16:46:36.303 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f4056441700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c77a0 con 0x7f4050069380 2022-11-27T16:46:36.303 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f4056441700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c7a20 con 0x7f40500cd0d0 2022-11-27T16:46:36.304 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050069380 0x7f40500ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.304 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050069380 0x7f40500ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56214/0 (socket says 172.21.15.201:56214) 2022-11-27T16:46:36.304 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 -- 172.21.15.201:0/2204371425 learned_addr learned my addr 172.21.15.201:0/2204371425 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:36.304 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404f7fe700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f40500cd0d0 0x7f40500c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.305 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f40549de700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4050156120 0x7f4050155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.305 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 -- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f40500cd0d0 msgr2=0x7f40500c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.305 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f40500cd0d0 0x7f40500c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.305 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 -- 172.21.15.201:0/2204371425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4050156120 msgr2=0x7f4050155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.306 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4050156120 0x7f4050155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.306 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 -- 172.21.15.201:0/2204371425 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4040009500 con 0x7f4050069380 2022-11-27T16:46:36.306 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404ffff700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050069380 0x7f40500ca320 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4040009c10 tx=0x7f4040004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.306 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404cff9700 1 -- 172.21.15.201:0/2204371425 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040004580 con 0x7f4050069380 2022-11-27T16:46:36.307 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404cff9700 1 -- 172.21.15.201:0/2204371425 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040011420 con 0x7f4050069380 2022-11-27T16:46:36.307 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404cff9700 1 -- 172.21.15.201:0/2204371425 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400117d0 con 0x7f4050069380 2022-11-27T16:46:36.307 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500c7d00 con 0x7f4050069380 2022-11-27T16:46:36.307 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40500c82c0 con 0x7f4050069380 2022-11-27T16:46:36.307 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404cff9700 1 -- 172.21.15.201:0/2204371425 <== mon.1 v2:172.21.15.201:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f404002c070 con 0x7f4050069380 2022-11-27T16:46:36.308 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.293+0000 7f404cff9700 1 -- 172.21.15.201:0/2204371425 <== mon.1 v2:172.21.15.201:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f404001c990 con 0x7f4050069380 2022-11-27T16:46:36.308 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.294+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40500739b0 con 0x7f4050069380 2022-11-27T16:46:36.308 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.296+0000 7f404cff9700 1 -- 172.21.15.201:0/2204371425 <== mon.1 v2:172.21.15.201:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f404002a060 con 0x7f4050069380 2022-11-27T16:46:36.371 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:36.369+0000 7f4c500ad200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:46:36.379 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.378+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f405015c960 con 0x7f4050069380 2022-11-27T16:46:36.387 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.385+0000 7f404cff9700 1 -- 172.21.15.201:0/2204371425 <== mon.1 v2:172.21.15.201:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f40400340b0 con 0x7f4050069380 2022-11-27T16:46:36.387 INFO:teuthology.orchestra.run.smithi201.stdout:5 2022-11-27T16:46:36.389 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.386+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050069380 msgr2=0x7f40500ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.389 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.386+0000 7f4056441700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050069380 0x7f40500ca320 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4040009c10 tx=0x7f4040004750 comp rx=0 tx=0).stop 2022-11-27T16:46:36.389 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.386+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 shutdown_connections 2022-11-27T16:46:36.389 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.386+0000 7f4056441700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f4050156120 0x7f4050155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.390 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.386+0000 7f4056441700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f4050069380 0x7f40500ca320 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.390 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.387+0000 7f4056441700 1 --2- 172.21.15.201:0/2204371425 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f40500cd0d0 0x7f40500c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.390 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.387+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 >> 172.21.15.201:0/2204371425 conn(0x7f405006a490 msgr2=0x7f405015ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.390 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.387+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 shutdown_connections 2022-11-27T16:46:36.391 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.387+0000 7f4056441700 1 -- 172.21.15.201:0/2204371425 wait complete. 2022-11-27T16:46:36.400 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2022-11-27T16:46:36.489 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:36.487+0000 7f8e2cc53200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2022-11-27T16:46:36.521 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.519+0000 7f0950b2c700 1 Processor -- start 2022-11-27T16:46:36.521 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.519+0000 7f0950b2c700 1 -- start start 2022-11-27T16:46:36.531 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f0950b2c700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f094c15c370 0x7f094c0af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.532 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f0950b2c700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f094c0d0430 0x7f094c0d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.532 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f0950b2c700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0b0c90 0x7f094c0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.532 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f0950b2c700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094c0d0e10 con 0x7f094c0d0430 2022-11-27T16:46:36.533 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f0950b2c700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094c0af970 con 0x7f094c15c370 2022-11-27T16:46:36.533 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f0950b2c700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0afbf0 con 0x7f094c0b0c90 2022-11-27T16:46:36.533 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f094a59c700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f094c15c370 0x7f094c0af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.533 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.520+0000 7f094ad9d700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0b0c90 0x7f094c0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.534 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094ad9d700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0b0c90 0x7f094c0b1110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38922/0 (socket says 172.21.15.201:38922) 2022-11-27T16:46:36.534 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094a59c700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f094c15c370 0x7f094c0af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56220/0 (socket says 172.21.15.201:56220) 2022-11-27T16:46:36.534 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094ad9d700 1 -- 172.21.15.201:0/1530506674 learned_addr learned my addr 172.21.15.201:0/1530506674 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:36.534 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f0949d9b700 1 --2- 172.21.15.201:0/1530506674 >> v2:172.21.15.145:3300/0 conn(0x7f094c0d0430 0x7f094c0d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.534 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094a59c700 1 -- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0b0c90 msgr2=0x7f094c0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:36.535 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094a59c700 1 --2- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0b0c90 0x7f094c0b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.535 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094a59c700 1 -- 172.21.15.201:0/1530506674 >> v2:172.21.15.145:3300/0 conn(0x7f094c0d0430 msgr2=0x7f094c0d08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:36.535 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094a59c700 1 --2- 172.21.15.201:0/1530506674 >> v2:172.21.15.145:3300/0 conn(0x7f094c0d0430 0x7f094c0d08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.535 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094a59c700 1 -- 172.21.15.201:0/1530506674 --> v2:172.21.15.201:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0afed0 con 0x7f094c15c370 2022-11-27T16:46:36.536 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094ad9d700 1 --2- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0b0c90 0x7f094c0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:36.536 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094a59c700 1 --2- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c15c370 0x7f094c0af430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f093c009820 tx=0x7f093c009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41412f0937ea7d22 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.536 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094959a700 1 -- 172.21.15.201:0/1530506674 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c005e30 con 0x7f094c15c370 2022-11-27T16:46:36.536 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094959a700 1 -- 172.21.15.201:0/1530506674 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c006030 con 0x7f094c15c370 2022-11-27T16:46:36.537 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.521+0000 7f094959a700 1 -- 172.21.15.201:0/1530506674 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c0063c0 con 0x7f094c15c370 2022-11-27T16:46:36.537 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 -- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c15c370 msgr2=0x7f094c0af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.537 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 --2- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c15c370 0x7f094c0af430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f093c009820 tx=0x7f093c009b30 comp rx=0 tx=0).stop 2022-11-27T16:46:36.538 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 -- 172.21.15.201:0/1530506674 shutdown_connections 2022-11-27T16:46:36.538 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 --2- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c15c370 0x7f094c0af430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.538 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 --2- 172.21.15.201:0/1530506674 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0b0c90 0x7f094c0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.538 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 --2- 172.21.15.201:0/1530506674 >> v2:172.21.15.145:3300/0 conn(0x7f094c0d0430 0x7f094c0d08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.538 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 -- 172.21.15.201:0/1530506674 >> 172.21.15.201:0/1530506674 conn(0x7f094c09bbd0 msgr2=0x7f094c0d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.539 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 -- 172.21.15.201:0/1530506674 shutdown_connections 2022-11-27T16:46:36.539 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.522+0000 7f0950b2c700 1 -- 172.21.15.201:0/1530506674 wait complete. 2022-11-27T16:46:36.539 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.523+0000 7f0950b2c700 1 Processor -- start 2022-11-27T16:46:36.540 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.523+0000 7f0950b2c700 1 -- start start 2022-11-27T16:46:36.540 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f0950b2c700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c0b0c90 0x7f094c157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.540 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f0950b2c700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0d0430 0x7f094c0b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.540 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f0950b2c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f094c15c370 0x7f094c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.540 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f0950b2c700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0b4d80 con 0x7f094c15c370 2022-11-27T16:46:36.541 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f0950b2c700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c158300 con 0x7f094c0b0c90 2022-11-27T16:46:36.541 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f0950b2c700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0b22c0 con 0x7f094c0d0430 2022-11-27T16:46:36.541 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c0b0c90 0x7f094c157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.541 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c0b0c90 0x7f094c157dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56226/0 (socket says 172.21.15.201:56226) 2022-11-27T16:46:36.542 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 -- 172.21.15.201:0/2090286422 learned_addr learned my addr 172.21.15.201:0/2090286422 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:36.542 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f0949d9b700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0d0430 0x7f094c0b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.542 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094ad9d700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f094c15c370 0x7f094c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.542 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 -- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0d0430 msgr2=0x7f094c0b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.543 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0d0430 0x7f094c0b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.543 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 -- 172.21.15.201:0/2090286422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f094c15c370 msgr2=0x7f094c1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.543 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f094c15c370 0x7f094c1584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.543 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.524+0000 7f094a59c700 1 -- 172.21.15.201:0/2090286422 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944009b30 con 0x7f094c0b0c90 2022-11-27T16:46:36.544 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f094a59c700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c0b0c90 0x7f094c157dc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f093c009b70 tx=0x7f093c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.544 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f093affd700 1 -- 172.21.15.201:0/2090286422 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c0063a0 con 0x7f094c0b0c90 2022-11-27T16:46:36.544 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f093affd700 1 -- 172.21.15.201:0/2090286422 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c006500 con 0x7f094c0b0c90 2022-11-27T16:46:36.545 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093c009430 con 0x7f094c0b0c90 2022-11-27T16:46:36.545 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f093affd700 1 -- 172.21.15.201:0/2090286422 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c0068b0 con 0x7f094c0b0c90 2022-11-27T16:46:36.546 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c0b27f0 con 0x7f094c0b0c90 2022-11-27T16:46:36.546 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f093affd700 1 -- 172.21.15.201:0/2090286422 <== mon.1 v2:172.21.15.201:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f093c00f050 con 0x7f094c0b0c90 2022-11-27T16:46:36.547 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.525+0000 7f093affd700 1 -- 172.21.15.201:0/2090286422 <== mon.1 v2:172.21.15.201:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f093c025d80 con 0x7f094c0b0c90 2022-11-27T16:46:36.547 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.526+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c0739b0 con 0x7f094c0b0c90 2022-11-27T16:46:36.548 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.528+0000 7f093affd700 1 -- 172.21.15.201:0/2090286422 <== mon.1 v2:172.21.15.201:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f093c003b20 con 0x7f094c0b0c90 2022-11-27T16:46:36.612 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.611+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7f094c0b5cf0 con 0x7f094c0b0c90 2022-11-27T16:46:36.627 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.625+0000 7f093affd700 1 -- 172.21.15.201:0/2090286422 <== mon.1 v2:172.21.15.201:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f093c003cc0 con 0x7f094c0b0c90 2022-11-27T16:46:36.628 INFO:teuthology.orchestra.run.smithi201.stdout:6 2022-11-27T16:46:36.630 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c0b0c90 msgr2=0x7f094c157dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.631 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c0b0c90 0x7f094c157dc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f093c009b70 tx=0x7f093c004750 comp rx=0 tx=0).stop 2022-11-27T16:46:36.631 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 shutdown_connections 2022-11-27T16:46:36.631 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f094c15c370 0x7f094c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.631 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f094c0b0c90 0x7f094c157dc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.632 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 --2- 172.21.15.201:0/2090286422 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f094c0d0430 0x7f094c0b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.632 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 >> 172.21.15.201:0/2090286422 conn(0x7f094c09bbd0 msgr2=0x7f094c15b650 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.632 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 shutdown_connections 2022-11-27T16:46:36.632 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.626+0000 7f0950b2c700 1 -- 172.21.15.201:0/2090286422 wait complete. 2022-11-27T16:46:36.640 DEBUG:teuthology.orchestra.run.smithi201:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2022-11-27T16:46:36.761 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.760+0000 7f531eba1700 1 Processor -- start 2022-11-27T16:46:36.762 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.760+0000 7f531eba1700 1 -- start start 2022-11-27T16:46:36.768 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531eba1700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f53180cb7b0 0x7f531815d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.769 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531eba1700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53180cd240 con 0x7f53180cc500 2022-11-27T16:46:36.769 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531eba1700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5318070530 con 0x7f5318158dd0 2022-11-27T16:46:36.769 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531eba1700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f531815a2d0 con 0x7f53180cb7b0 2022-11-27T16:46:36.769 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531db9f700 1 --1- >> v1:172.21.15.201:6789/0 conn(0x7f5318158dd0 0x7f53180cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.201:6789/0 says I am v1:172.21.15.201:56534/0 (socket says 172.21.15.201:56534) 2022-11-27T16:46:36.770 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531db9f700 1 -- 172.21.15.201:0/4282607184 learned_addr learned my addr 172.21.15.201:0/4282607184 (peer_addr_for_me v1:172.21.15.201:0/0) 2022-11-27T16:46:36.770 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531e3a0700 1 --2- 172.21.15.201:0/4282607184 >> v2:172.21.15.145:3300/0 conn(0x7f53180cb7b0 0x7f531815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.770 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 <== mon.1 v1:172.21.15.201:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820070882 0 0) 0x7f5318070530 con 0x7f5318158dd0 2022-11-27T16:46:36.770 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 --> v1:172.21.15.201:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53000035b0 con 0x7f5318158dd0 2022-11-27T16:46:36.771 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1633682856 0 0) 0x7f53180cd240 con 0x7f53180cc500 2022-11-27T16:46:36.771 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5318070530 con 0x7f53180cc500 2022-11-27T16:46:36.771 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 <== mon.1 v1:172.21.15.201:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1963469436 0 0) 0x7f53000035b0 con 0x7f5318158dd0 2022-11-27T16:46:36.771 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 >> v2:172.21.15.145:3300/0 conn(0x7f53180cb7b0 msgr2=0x7f531815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:36.772 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 --2- 172.21.15.201:0/4282607184 >> v2:172.21.15.145:3300/0 conn(0x7f53180cb7b0 0x7f531815d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.772 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 >> v1:172.21.15.145:6789/0 conn(0x7f53180cc500 legacy=0x7f53180cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.772 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 --> v1:172.21.15.201:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5318159510 con 0x7f5318158dd0 2022-11-27T16:46:36.772 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 <== mon.1 v1:172.21.15.201:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1749869138 0 0) 0x7f53140016b0 con 0x7f5318158dd0 2022-11-27T16:46:36.773 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531e3a0700 1 --2- 172.21.15.201:0/4282607184 >> v2:172.21.15.145:3300/0 conn(0x7f53180cb7b0 0x7f531815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:36.773 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 >> v1:172.21.15.201:6789/0 conn(0x7f5318158dd0 legacy=0x7f53180cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.773 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5300003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.773 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5300004850 0x7f5300004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.773 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f5300005200 0x7f530000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.774 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531815a2d0 con 0x7f5300004850 2022-11-27T16:46:36.774 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530000e270 con 0x7f5300005200 2022-11-27T16:46:36.774 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530000e430 con 0x7f5300003b90 2022-11-27T16:46:36.774 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531db9f700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5300003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.775 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531e3a0700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f5300005200 0x7f530000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.775 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.761+0000 7f531d39e700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5300004850 0x7f5300004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.775 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531db9f700 1 -- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f5300005200 msgr2=0x7f530000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:36.775 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531db9f700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f5300005200 0x7f530000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.775 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531db9f700 1 -- 172.21.15.201:0/4282607184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5300004850 msgr2=0x7f5300004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:36.776 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531db9f700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5300004850 0x7f5300004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.776 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531db9f700 1 -- 172.21.15.201:0/4282607184 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000035b0 con 0x7f5300003b90 2022-11-27T16:46:36.776 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531e3a0700 1 -- 172.21.15.201:0/4282607184 reap_dead start 2022-11-27T16:46:36.776 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531db9f700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5300003f80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f531400c240 tx=0x7f5300005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d087952baa1471 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.777 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531400cd80 con 0x7f5300003b90 2022-11-27T16:46:36.777 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314007b10 con 0x7f5300003b90 2022-11-27T16:46:36.777 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.762+0000 7f531cb9d700 1 -- 172.21.15.201:0/4282607184 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314005ba0 con 0x7f5300003b90 2022-11-27T16:46:36.777 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.763+0000 7f531eba1700 1 -- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 msgr2=0x7f5300003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.778 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.763+0000 7f531eba1700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5300003f80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f531400c240 tx=0x7f5300005880 comp rx=0 tx=0).stop 2022-11-27T16:46:36.778 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.763+0000 7f531eba1700 1 -- 172.21.15.201:0/4282607184 shutdown_connections 2022-11-27T16:46:36.778 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.763+0000 7f531eba1700 1 --2- 172.21.15.201:0/4282607184 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5300003f80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.778 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.763+0000 7f531eba1700 1 -- 172.21.15.201:0/4282607184 >> 172.21.15.201:0/4282607184 conn(0x7f531806a490 msgr2=0x7f531806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.778 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.763+0000 7f531eba1700 1 -- 172.21.15.201:0/4282607184 shutdown_connections 2022-11-27T16:46:36.779 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.763+0000 7f531eba1700 1 -- 172.21.15.201:0/4282607184 wait complete. 2022-11-27T16:46:36.779 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.764+0000 7f531eba1700 1 Processor -- start 2022-11-27T16:46:36.779 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.764+0000 7f531eba1700 1 -- start start 2022-11-27T16:46:36.779 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5318148500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.779 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5318158130 0x7f53180b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.780 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f53180b2210 0x7f53180af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:36.780 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5318148370 con 0x7f5318158130 2022-11-27T16:46:36.780 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531814af50 con 0x7f53180b2210 2022-11-27T16:46:36.780 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53181585a0 con 0x7f5300003b90 2022-11-27T16:46:36.781 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531db9f700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5318148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.781 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f53180b2210 0x7f53180af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.781 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f53180b2210 0x7f53180af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.201:56248/0 (socket says 172.21.15.201:56248) 2022-11-27T16:46:36.781 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 -- 172.21.15.201:0/2220111730 learned_addr learned my addr 172.21.15.201:0/2220111730 (peer_addr_for_me v2:172.21.15.201:0/0) 2022-11-27T16:46:36.782 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531db9f700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5318148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.201:38942/0 (socket says 172.21.15.201:38942) 2022-11-27T16:46:36.782 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531d39e700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5318158130 0x7f53180b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:36.782 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 -- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 msgr2=0x7f5318148500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.782 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5318148500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.783 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 -- 172.21.15.201:0/2220111730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5318158130 msgr2=0x7f53180b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.783 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5318158130 0x7f53180b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.783 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 -- 172.21.15.201:0/2220111730 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f531814a340 con 0x7f53180b2210 2022-11-27T16:46:36.783 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531db9f700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5318148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:36.783 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531e3a0700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f53180b2210 0x7f53180af430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53180ceec0 tx=0x7f531000f490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:36.784 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f53067fc700 1 -- 172.21.15.201:0/2220111730 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531001c070 con 0x7f53180b2210 2022-11-27T16:46:36.784 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f53067fc700 1 -- 172.21.15.201:0/2220111730 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f531000f040 con 0x7f53180b2210 2022-11-27T16:46:36.784 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531814a620 con 0x7f53180b2210 2022-11-27T16:46:36.784 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f53067fc700 1 -- 172.21.15.201:0/2220111730 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531000e040 con 0x7f53180b2210 2022-11-27T16:46:36.784 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531814abe0 con 0x7f53180b2210 2022-11-27T16:46:36.785 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f53067fc700 1 -- 172.21.15.201:0/2220111730 <== mon.1 v2:172.21.15.201:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f5310003a20 con 0x7f53180b2210 2022-11-27T16:46:36.785 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.765+0000 7f53067fc700 1 -- 172.21.15.201:0/2220111730 <== mon.1 v2:172.21.15.201:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f5310015430 con 0x7f53180b2210 2022-11-27T16:46:36.785 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.766+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53180cefc0 con 0x7f53180b2210 2022-11-27T16:46:36.786 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.768+0000 7f53067fc700 1 -- 172.21.15.201:0/2220111730 <== mon.1 v2:172.21.15.201:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f531000fdc0 con 0x7f53180b2210 2022-11-27T16:46:36.852 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.851+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f531804e190 con 0x7f53180b2210 2022-11-27T16:46:36.858 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.857+0000 7f53067fc700 1 -- 172.21.15.201:0/2220111730 <== mon.1 v2:172.21.15.201:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f53100300b0 con 0x7f53180b2210 2022-11-27T16:46:36.859 INFO:teuthology.orchestra.run.smithi201.stdout:7 2022-11-27T16:46:36.860 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f53180b2210 msgr2=0x7f53180af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:36.860 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f53180b2210 0x7f53180af430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f53180ceec0 tx=0x7f531000f490 comp rx=0 tx=0).stop 2022-11-27T16:46:36.861 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 shutdown_connections 2022-11-27T16:46:36.861 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f53180b2210 0x7f53180af430 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.861 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f5300003b90 0x7f5318148500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.861 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 --2- 172.21.15.201:0/2220111730 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f5318158130 0x7f53180b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:36.862 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 >> 172.21.15.201:0/2220111730 conn(0x7f531806a490 msgr2=0x7f53180cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:36.862 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 shutdown_connections 2022-11-27T16:46:36.862 INFO:teuthology.orchestra.run.smithi201.stderr:2022-11-27T16:46:36.858+0000 7f531eba1700 1 -- 172.21.15.201:0/2220111730 wait complete. 2022-11-27T16:46:36.871 INFO:tasks.ceph.osd.0:Restarting daemon 2022-11-27T16:46:36.872 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2022-11-27T16:46:36.875 INFO:tasks.ceph.osd.0:Started 2022-11-27T16:46:36.876 INFO:tasks.ceph.osd.1:Restarting daemon 2022-11-27T16:46:36.877 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2022-11-27T16:46:36.880 INFO:tasks.ceph.osd.1:Started 2022-11-27T16:46:36.880 INFO:tasks.ceph.osd.2:Restarting daemon 2022-11-27T16:46:36.881 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2022-11-27T16:46:36.883 INFO:tasks.ceph.osd.2:Started 2022-11-27T16:46:36.884 INFO:tasks.ceph.osd.3:Restarting daemon 2022-11-27T16:46:36.884 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2022-11-27T16:46:36.887 INFO:tasks.ceph.osd.3:Started 2022-11-27T16:46:36.887 INFO:tasks.ceph.osd.4:Restarting daemon 2022-11-27T16:46:36.887 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2022-11-27T16:46:36.890 INFO:tasks.ceph.osd.4:Started 2022-11-27T16:46:36.890 INFO:tasks.ceph.osd.5:Restarting daemon 2022-11-27T16:46:36.891 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2022-11-27T16:46:36.893 INFO:tasks.ceph.osd.5:Started 2022-11-27T16:46:36.893 INFO:tasks.ceph.osd.6:Restarting daemon 2022-11-27T16:46:36.894 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2022-11-27T16:46:36.896 INFO:tasks.ceph.osd.6:Started 2022-11-27T16:46:36.896 INFO:tasks.ceph.osd.7:Restarting daemon 2022-11-27T16:46:36.897 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2022-11-27T16:46:36.899 INFO:tasks.ceph.osd.7:Started 2022-11-27T16:46:36.900 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:46:36.951 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:36.949+0000 7f4c500ad200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:46:36.968 INFO:tasks.ceph.osd.1.smithi145.stderr:2022-11-27T16:46:36.966+0000 7f92fff6e5c0 -1 Falling back to public interface 2022-11-27T16:46:36.983 INFO:tasks.ceph.osd.1.smithi145.stderr:2022-11-27T16:46:36.981+0000 7f92fff6e5c0 -1 osd.1 0 log_to_monitors true 2022-11-27T16:46:36.989 INFO:tasks.ceph.osd.0.smithi145.stderr:2022-11-27T16:46:36.987+0000 7fd24e93f5c0 -1 Falling back to public interface 2022-11-27T16:46:36.991 INFO:tasks.ceph.osd.2.smithi145.stderr:2022-11-27T16:46:36.989+0000 7fd2d7f3e5c0 -1 Falling back to public interface 2022-11-27T16:46:36.992 INFO:tasks.ceph.osd.0.smithi145.stderr:2022-11-27T16:46:36.989+0000 7fd245cfd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6814/0: (98) Address already in use 2022-11-27T16:46:36.996 INFO:tasks.ceph.osd.4.smithi201.stderr:2022-11-27T16:46:36.995+0000 7f762ec755c0 -1 Falling back to public interface 2022-11-27T16:46:36.997 INFO:tasks.ceph.osd.5.smithi201.stderr:2022-11-27T16:46:36.996+0000 7f2850fa25c0 -1 Falling back to public interface 2022-11-27T16:46:37.001 INFO:tasks.ceph.osd.6.smithi201.stderr:2022-11-27T16:46:37.000+0000 7f207a1ea5c0 -1 Falling back to public interface 2022-11-27T16:46:37.006 INFO:tasks.ceph.osd.0.smithi145.stderr:2022-11-27T16:46:37.004+0000 7fd24e93f5c0 -1 osd.0 0 log_to_monitors true 2022-11-27T16:46:37.010 INFO:tasks.ceph.osd.2.smithi145.stderr:2022-11-27T16:46:37.007+0000 7fd2d7f3e5c0 -1 osd.2 0 log_to_monitors true 2022-11-27T16:46:37.011 INFO:tasks.ceph.osd.4.smithi201.stderr:2022-11-27T16:46:37.010+0000 7f762ec755c0 -1 osd.4 0 log_to_monitors true 2022-11-27T16:46:37.013 INFO:tasks.ceph.osd.3.smithi145.stderr:2022-11-27T16:46:37.010+0000 7fc79b7445c0 -1 Falling back to public interface 2022-11-27T16:46:37.014 INFO:tasks.ceph.osd.5.smithi201.stderr:2022-11-27T16:46:37.012+0000 7f2850fa25c0 -1 osd.5 0 log_to_monitors true 2022-11-27T16:46:37.020 INFO:tasks.ceph.osd.6.smithi201.stderr:2022-11-27T16:46:37.019+0000 7f207a1ea5c0 -1 osd.6 0 log_to_monitors true 2022-11-27T16:46:37.029 INFO:tasks.ceph.osd.3.smithi145.stderr:2022-11-27T16:46:37.026+0000 7fc79b7445c0 -1 osd.3 0 log_to_monitors true 2022-11-27T16:46:37.033 INFO:tasks.ceph.osd.7.smithi201.stderr:2022-11-27T16:46:37.031+0000 7f793dead5c0 -1 Falling back to public interface 2022-11-27T16:46:37.047 INFO:tasks.ceph.osd.7.smithi201.stderr:2022-11-27T16:46:37.045+0000 7f793dead5c0 -1 osd.7 0 log_to_monitors true 2022-11-27T16:46:37.055 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.052+0000 7ffa52074700 1 Processor -- start 2022-11-27T16:46:37.055 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.052+0000 7ffa52074700 1 -- start start 2022-11-27T16:46:37.058 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa52074700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffa4c0ce9a0 0x7ffa4c0af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:37.059 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa52074700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7ffa4c0aece0 0x7ffa4c0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:37.059 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa52074700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c0b0550 con 0x7ffa4c15a0d0 2022-11-27T16:46:37.059 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa52074700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c0af100 con 0x7ffa4c0ce9a0 2022-11-27T16:46:37.059 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa52074700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa4c0af3a0 con 0x7ffa4c0aece0 2022-11-27T16:46:37.060 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa43fff700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffa4c0ce9a0 0x7ffa4c0af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:37.060 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa43fff700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7ffa4c0ce9a0 0x7ffa4c0af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54420/0 (socket says 172.21.15.145:54420) 2022-11-27T16:46:37.060 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa43fff700 1 -- 172.21.15.145:0/1871402083 learned_addr learned my addr 172.21.15.145:0/1871402083 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:37.060 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa4bfff700 1 --2- 172.21.15.145:0/1871402083 >> v2:172.21.15.201:3300/0 conn(0x7ffa4c0aece0 0x7ffa4c0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:37.061 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa43fff700 1 -- 172.21.15.145:0/1871402083 >> v2:172.21.15.201:3300/0 conn(0x7ffa4c0aece0 msgr2=0x7ffa4c0b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:37.061 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa43fff700 1 --2- 172.21.15.145:0/1871402083 >> v2:172.21.15.201:3300/0 conn(0x7ffa4c0aece0 0x7ffa4c0b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.061 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa43fff700 1 -- 172.21.15.145:0/1871402083 >> v1:172.21.15.201:6789/0 conn(0x7ffa4c15a0d0 legacy=0x7ffa4c0ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:37.061 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.053+0000 7ffa43fff700 1 -- 172.21.15.145:0/1871402083 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c0afc20 con 0x7ffa4c0ce9a0 2022-11-27T16:46:37.061 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa43fff700 1 --2- 172.21.15.145:0/1871402083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c0af610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c069e40 tx=0x7ffa34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9eb153e49ef6f8 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:37.062 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa4affd700 1 -- 172.21.15.145:0/1871402083 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34011830 con 0x7ffa4c0ce9a0 2022-11-27T16:46:37.062 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa4affd700 1 -- 172.21.15.145:0/1871402083 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa34011990 con 0x7ffa4c0ce9a0 2022-11-27T16:46:37.062 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa4affd700 1 -- 172.21.15.145:0/1871402083 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34011d20 con 0x7ffa4c0ce9a0 2022-11-27T16:46:37.062 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa52074700 1 -- 172.21.15.145:0/1871402083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0ce9a0 msgr2=0x7ffa4c0af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:37.063 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa52074700 1 --2- 172.21.15.145:0/1871402083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c0af610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c069e40 tx=0x7ffa34009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.063 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa52074700 1 -- 172.21.15.145:0/1871402083 shutdown_connections 2022-11-27T16:46:37.063 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa52074700 1 --2- 172.21.15.145:0/1871402083 >> v2:172.21.15.201:3300/0 conn(0x7ffa4c0aece0 0x7ffa4c0b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.063 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa52074700 1 --2- 172.21.15.145:0/1871402083 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c0af610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.064 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.054+0000 7ffa52074700 1 -- 172.21.15.145:0/1871402083 >> 172.21.15.145:0/1871402083 conn(0x7ffa4c099a00 msgr2=0x7ffa4c0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:37.064 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.055+0000 7ffa52074700 1 -- 172.21.15.145:0/1871402083 shutdown_connections 2022-11-27T16:46:37.064 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.055+0000 7ffa52074700 1 -- 172.21.15.145:0/1871402083 wait complete. 2022-11-27T16:46:37.064 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.056+0000 7ffa52074700 1 Processor -- start 2022-11-27T16:46:37.065 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.056+0000 7ffa52074700 1 -- start start 2022-11-27T16:46:37.070 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.056+0000 7ffa52074700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0aece0 0x7ffa4c1f1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:37.070 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.056+0000 7ffa52074700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c06aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:37.071 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.056+0000 7ffa52074700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7ffa4c15a0d0 0x7ffa4c1f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:37.071 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.056+0000 7ffa52074700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1f1c40 con 0x7ffa4c0aece0 2022-11-27T16:46:37.071 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.056+0000 7ffa52074700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1f2720 con 0x7ffa4c0ce9a0 2022-11-27T16:46:37.072 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa52074700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c06b400 con 0x7ffa4c15a0d0 2022-11-27T16:46:37.072 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0aece0 0x7ffa4c1f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:37.072 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c06aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:37.073 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4bfff700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7ffa4c15a0d0 0x7ffa4c1f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:37.073 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0aece0 0x7ffa4c1f1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54424/0 (socket says 172.21.15.145:54424) 2022-11-27T16:46:37.073 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 -- 172.21.15.145:0/2894892529 learned_addr learned my addr 172.21.15.145:0/2894892529 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:37.073 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa43fff700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c06aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3300/0 says I am v2:172.21.15.145:60958/0 (socket says 172.21.15.145:60958) 2022-11-27T16:46:37.073 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 -- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7ffa4c15a0d0 msgr2=0x7ffa4c1f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:37.074 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7ffa4c15a0d0 0x7ffa4c1f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.074 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 -- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7ffa4c0ce9a0 msgr2=0x7ffa4c06aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:37.074 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c06aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.074 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 -- 172.21.15.145:0/2894892529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34009550 con 0x7ffa4c0aece0 2022-11-27T16:46:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4bfff700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7ffa4c15a0d0 0x7ffa4c1f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa43fff700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c06aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa4b7fe700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0aece0 0x7ffa4c1f1700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00c4d0 tx=0x7ffa3c00c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa48ff9700 1 -- 172.21.15.145:0/2894892529 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0052d0 con 0x7ffa4c0aece0 2022-11-27T16:46:37.075 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa48ff9700 1 -- 172.21.15.145:0/2894892529 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c005430 con 0x7ffa4c0aece0 2022-11-27T16:46:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c1f1e80 con 0x7ffa4c0aece0 2022-11-27T16:46:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa48ff9700 1 -- 172.21.15.145:0/2894892529 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0038f0 con 0x7ffa4c0aece0 2022-11-27T16:46:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c1f2440 con 0x7ffa4c0aece0 2022-11-27T16:46:37.076 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.057+0000 7ffa48ff9700 1 -- 172.21.15.145:0/2894892529 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ffa3c019030 con 0x7ffa4c0aece0 2022-11-27T16:46:37.077 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.058+0000 7ffa48ff9700 1 -- 172.21.15.145:0/2894892529 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7ffa3c011df0 con 0x7ffa4c0aece0 2022-11-27T16:46:37.077 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.058+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c0739b0 con 0x7ffa4c0aece0 2022-11-27T16:46:37.077 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.060+0000 7ffa48ff9700 1 -- 172.21.15.145:0/2894892529 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7ffa3c017070 con 0x7ffa4c0aece0 2022-11-27T16:46:37.098 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:37.096+0000 7f8e2cc53200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2022-11-27T16:46:37.134 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.131+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa4c0c7530 con 0x7ffa4c0aece0 2022-11-27T16:46:37.135 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.131+0000 7ffa48ff9700 1 -- 172.21.15.145:0/2894892529 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7011 (secure 0 0 0) 0x7ffa3c003a50 con 0x7ffa4c0aece0 2022-11-27T16:46:37.136 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:37.136 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":10,"fsid":"0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7","created":"2022-11-27T16:46:34.409983+0000","modified":"2022-11-27T16:46:36.852469+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:46:36.852469+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:46:37.137 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0aece0 msgr2=0x7ffa4c1f1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:37.138 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0aece0 0x7ffa4c1f1700 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00c4d0 tx=0x7ffa3c00c7e0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.138 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 shutdown_connections 2022-11-27T16:46:37.138 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7ffa4c15a0d0 0x7ffa4c1f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7ffa4c0aece0 0x7ffa4c1f1700 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 --2- 172.21.15.145:0/2894892529 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7ffa4c0ce9a0 0x7ffa4c06aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 >> 172.21.15.145:0/2894892529 conn(0x7ffa4c099a00 msgr2=0x7ffa4c06f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:37.139 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 shutdown_connections 2022-11-27T16:46:37.140 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:37.132+0000 7ffa52074700 1 -- 172.21.15.145:0/2894892529 wait complete. 2022-11-27T16:46:37.140 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:37.134+0000 7f4c500ad200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:46:37.147 INFO:tasks.ceph.ceph_manager.ceph:[] 2022-11-27T16:46:37.148 INFO:tasks.ceph:Waiting for OSDs to come up 2022-11-27T16:46:37.251 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:37.250+0000 7f8e2cc53200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2022-11-27T16:46:37.816 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:37.813+0000 7f4c500ad200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:46:37.892 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:37.889+0000 7f4c500ad200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:46:37.916 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:37.915+0000 7f8e2cc53200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:46:37.953 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:46:37.991 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:37.990+0000 7f8e2cc53200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2022-11-27T16:46:38.030 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:38.027+0000 7f4c500ad200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:46:38.066 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.063+0000 7f8a5cff1700 1 Processor -- start 2022-11-27T16:46:38.067 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.064+0000 7f8a5cff1700 1 -- start start 2022-11-27T16:46:38.072 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.064+0000 7f8a5cff1700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580cb1f0 0x7f8a58071450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:38.073 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.064+0000 7f8a5cff1700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f8a580693f0 0x7f8a58069890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:38.073 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5cff1700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5815c9e0 con 0x7f8a580706c0 2022-11-27T16:46:38.073 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5cff1700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a58069dd0 con 0x7f8a580693f0 2022-11-27T16:46:38.073 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5cff1700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5815d580 con 0x7f8a580cb1f0 2022-11-27T16:46:38.074 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580cb1f0 0x7f8a58071450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:38.074 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580cb1f0 0x7f8a58071450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.201:3301/0 says I am v2:172.21.15.145:56818/0 (socket says 172.21.15.145:56818) 2022-11-27T16:46:38.074 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 -- 172.21.15.145:0/2759692842 learned_addr learned my addr 172.21.15.145:0/2759692842 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:38.074 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a56d9d700 1 --2- 172.21.15.145:0/2759692842 >> v2:172.21.15.201:3300/0 conn(0x7f8a580693f0 0x7f8a58069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:38.074 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 -- 172.21.15.145:0/2759692842 >> v2:172.21.15.201:3300/0 conn(0x7f8a580693f0 msgr2=0x7f8a58069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:38.075 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 --2- 172.21.15.145:0/2759692842 >> v2:172.21.15.201:3300/0 conn(0x7f8a580693f0 0x7f8a58069890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.075 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 -- 172.21.15.145:0/2759692842 >> v1:172.21.15.201:6789/0 conn(0x7f8a580706c0 legacy=0x7f8a5815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:38.075 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 -- 172.21.15.145:0/2759692842 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5815c6b0 con 0x7f8a580cb1f0 2022-11-27T16:46:38.076 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5559a700 1 -- 172.21.15.145:0/2759692842 <== mon.1 v1:172.21.15.201:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466300662 0 0) 0x7f8a5815c9e0 con 0x7f8a580706c0 2022-11-27T16:46:38.076 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a56d9d700 1 --2- 172.21.15.145:0/2759692842 >> v2:172.21.15.201:3300/0 conn(0x7f8a580693f0 0x7f8a58069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:38.077 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5659c700 1 --2- 172.21.15.145:0/2759692842 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580cb1f0 0x7f8a58071450 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a480098f0 tx=0x7f8a48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e8b4fa9fe0b62e6 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:38.077 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.065+0000 7f8a5559a700 1 -- 172.21.15.145:0/2759692842 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48011830 con 0x7f8a580cb1f0 2022-11-27T16:46:38.077 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5559a700 1 -- 172.21.15.145:0/2759692842 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a48011990 con 0x7f8a580cb1f0 2022-11-27T16:46:38.077 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5559a700 1 -- 172.21.15.145:0/2759692842 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48011d20 con 0x7f8a580cb1f0 2022-11-27T16:46:38.077 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 -- 172.21.15.145:0/2759692842 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580cb1f0 msgr2=0x7f8a58071450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:38.078 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 --2- 172.21.15.145:0/2759692842 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580cb1f0 0x7f8a58071450 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8a480098f0 tx=0x7f8a48009c00 comp rx=0 tx=0).stop 2022-11-27T16:46:38.078 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 -- 172.21.15.145:0/2759692842 shutdown_connections 2022-11-27T16:46:38.078 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 --2- 172.21.15.145:0/2759692842 >> v2:172.21.15.201:3300/0 conn(0x7f8a580693f0 0x7f8a58069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.078 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 --2- 172.21.15.145:0/2759692842 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580cb1f0 0x7f8a58071450 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.079 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 -- 172.21.15.145:0/2759692842 >> 172.21.15.145:0/2759692842 conn(0x7f8a5806a500 msgr2=0x7f8a5806a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:38.079 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 -- 172.21.15.145:0/2759692842 shutdown_connections 2022-11-27T16:46:38.079 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.066+0000 7f8a5cff1700 1 -- 172.21.15.145:0/2759692842 wait complete. 2022-11-27T16:46:38.079 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.067+0000 7f8a5cff1700 1 Processor -- start 2022-11-27T16:46:38.080 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5cff1700 1 -- start start 2022-11-27T16:46:38.080 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5cff1700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8a580693f0 0x7f8a5814fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:38.080 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5cff1700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580706c0 0x7f8a580ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:38.080 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5cff1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a580cb1f0 0x7f8a580b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:38.080 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5cff1700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580ad560 con 0x7f8a580cb1f0 2022-11-27T16:46:38.081 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5cff1700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580ac990 con 0x7f8a580693f0 2022-11-27T16:46:38.081 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5cff1700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580acc30 con 0x7f8a580706c0 2022-11-27T16:46:38.081 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a56d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a580cb1f0 0x7f8a580b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:38.081 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a56d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a580cb1f0 0x7f8a580b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54438/0 (socket says 172.21.15.145:54438) 2022-11-27T16:46:38.082 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a56d9d700 1 -- 172.21.15.145:0/988406010 learned_addr learned my addr 172.21.15.145:0/988406010 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:38.082 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a5659c700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8a580693f0 0x7f8a5814fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:38.082 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.068+0000 7f8a55d9b700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580706c0 0x7f8a580ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:38.082 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a56d9d700 1 -- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580706c0 msgr2=0x7f8a580ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:38.083 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a56d9d700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580706c0 0x7f8a580ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.083 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a56d9d700 1 -- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8a580693f0 msgr2=0x7f8a5814fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:38.083 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a56d9d700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8a580693f0 0x7f8a5814fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.083 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a56d9d700 1 -- 172.21.15.145:0/988406010 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50009a70 con 0x7f8a580cb1f0 2022-11-27T16:46:38.084 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a5659c700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8a580693f0 0x7f8a5814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:38.084 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a55d9b700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580706c0 0x7f8a580ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2022-11-27T16:46:38.084 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a56d9d700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a580cb1f0 0x7f8a580b3440 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000efc0 tx=0x7f8a5000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:38.084 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a46ffd700 1 -- 172.21.15.145:0/988406010 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000ca90 con 0x7f8a580cb1f0 2022-11-27T16:46:38.085 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a46ffd700 1 -- 172.21.15.145:0/988406010 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5000f040 con 0x7f8a580cb1f0 2022-11-27T16:46:38.085 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a46ffd700 1 -- 172.21.15.145:0/988406010 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000ce30 con 0x7f8a580cb1f0 2022-11-27T16:46:38.085 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a48009550 con 0x7f8a580cb1f0 2022-11-27T16:46:38.085 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a580ad1f0 con 0x7f8a580cb1f0 2022-11-27T16:46:38.086 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a46ffd700 1 -- 172.21.15.145:0/988406010 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f8a50003a20 con 0x7f8a580cb1f0 2022-11-27T16:46:38.086 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.069+0000 7f8a46ffd700 1 -- 172.21.15.145:0/988406010 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1879+0+0 (secure 0 0 0) 0x7f8a50004700 con 0x7f8a580cb1f0 2022-11-27T16:46:38.086 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.070+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5804e190 con 0x7f8a580cb1f0 2022-11-27T16:46:38.087 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.072+0000 7f8a46ffd700 1 -- 172.21.15.145:0/988406010 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71351 (secure 0 0 0) 0x7f8a50018030 con 0x7f8a580cb1f0 2022-11-27T16:46:38.128 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:38.126+0000 7f8e2cc53200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2022-11-27T16:46:38.152 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.150+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a58070bf0 con 0x7f8a580cb1f0 2022-11-27T16:46:38.153 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.150+0000 7f8a46ffd700 1 -- 172.21.15.145:0/988406010 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7011 (secure 0 0 0) 0x7f8a50004020 con 0x7f8a580cb1f0 2022-11-27T16:46:38.154 INFO:teuthology.misc.health.smithi145.stdout: 2022-11-27T16:46:38.154 INFO:teuthology.misc.health.smithi145.stdout:{"epoch":11,"fsid":"0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7","created":"2022-11-27T16:46:34.409983+0000","modified":"2022-11-27T16:46:37.854795+0000","last_up_change":"0.000000","last_in_change":"2022-11-27T16:46:36.852469+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:46:38.155 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a580cb1f0 msgr2=0x7f8a580b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:38.156 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a580cb1f0 0x7f8a580b3440 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000efc0 tx=0x7f8a5000c3a0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.156 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 shutdown_connections 2022-11-27T16:46:38.156 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f8a580cb1f0 0x7f8a580b3440 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.156 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f8a580693f0 0x7f8a5814fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.157 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 --2- 172.21.15.145:0/988406010 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f8a580706c0 0x7f8a580ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:38.157 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 >> 172.21.15.145:0/988406010 conn(0x7f8a5806a500 msgr2=0x7f8a5815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:38.157 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 shutdown_connections 2022-11-27T16:46:38.157 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:38.152+0000 7f8a5cff1700 1 -- 172.21.15.145:0/988406010 wait complete. 2022-11-27T16:46:38.169 DEBUG:teuthology.misc:0 of 8 OSDs are up 2022-11-27T16:46:38.206 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:38.204+0000 7f4c500ad200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:46:38.285 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:38.283+0000 7f8e2cc53200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2022-11-27T16:46:38.336 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:38.333+0000 7f4c500ad200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:46:38.415 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:38.413+0000 7f8e2cc53200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2022-11-27T16:46:38.684 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:38.681+0000 7f4c500ad200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:46:38.763 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:38.762+0000 7f8e2cc53200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2022-11-27T16:46:38.834 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:38.831+0000 7f4c500ad200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:46:38.868 INFO:tasks.ceph.osd.2.smithi145.stderr:2022-11-27T16:46:38.865+0000 7fd2cc2f6700 -1 osd.2 0 waiting for initial osdmap 2022-11-27T16:46:38.869 INFO:tasks.ceph.osd.5.smithi201.stderr:2022-11-27T16:46:38.867+0000 7f284535a700 -1 osd.5 0 waiting for initial osdmap 2022-11-27T16:46:38.870 INFO:tasks.ceph.osd.5.smithi201.stderr:2022-11-27T16:46:38.869+0000 7f283e737700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.871 INFO:tasks.ceph.osd.2.smithi145.stderr:2022-11-27T16:46:38.869+0000 7fd2c56d3700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.873 INFO:tasks.ceph.osd.0.smithi145.stderr:2022-11-27T16:46:38.870+0000 7fd242cf7700 -1 osd.0 0 waiting for initial osdmap 2022-11-27T16:46:38.873 INFO:tasks.ceph.osd.1.smithi145.stderr:2022-11-27T16:46:38.870+0000 7f92f4326700 -1 osd.1 0 waiting for initial osdmap 2022-11-27T16:46:38.874 INFO:tasks.ceph.osd.3.smithi145.stderr:2022-11-27T16:46:38.871+0000 7fc78fafc700 -1 osd.3 0 waiting for initial osdmap 2022-11-27T16:46:38.874 INFO:tasks.ceph.osd.4.smithi201.stderr:2022-11-27T16:46:38.872+0000 7f762302d700 -1 osd.4 0 waiting for initial osdmap 2022-11-27T16:46:38.875 INFO:tasks.ceph.osd.6.smithi201.stderr:2022-11-27T16:46:38.872+0000 7f206e5a2700 -1 osd.6 0 waiting for initial osdmap 2022-11-27T16:46:38.875 INFO:tasks.ceph.osd.7.smithi201.stderr:2022-11-27T16:46:38.872+0000 7f7932265700 -1 osd.7 0 waiting for initial osdmap 2022-11-27T16:46:38.876 INFO:tasks.ceph.osd.0.smithi145.stderr:2022-11-27T16:46:38.873+0000 7fd23c0d4700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.876 INFO:tasks.ceph.osd.1.smithi145.stderr:2022-11-27T16:46:38.873+0000 7f92ed703700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.877 INFO:tasks.ceph.osd.3.smithi145.stderr:2022-11-27T16:46:38.874+0000 7fc788ed9700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.877 INFO:tasks.ceph.osd.4.smithi201.stderr:2022-11-27T16:46:38.875+0000 7f761c40a700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.878 INFO:tasks.ceph.osd.7.smithi201.stderr:2022-11-27T16:46:38.875+0000 7f792b642700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.878 INFO:tasks.ceph.osd.6.smithi201.stderr:2022-11-27T16:46:38.875+0000 7f206797f700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2022-11-27T16:46:38.925 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:38.923+0000 7f8e2cc53200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2022-11-27T16:46:38.958 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:38.955+0000 7f4c500ad200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:46:39.038 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:39.037+0000 7f8e2cc53200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2022-11-27T16:46:39.101 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:39.099+0000 7f4c500ad200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:46:39.181 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:39.179+0000 7f8e2cc53200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2022-11-27T16:46:39.200 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:39.197+0000 7f4c500ad200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:46:39.278 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:39.276+0000 7f8e2cc53200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2022-11-27T16:46:39.552 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:39.549+0000 7f4c500ad200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:46:39.637 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:39.635+0000 7f8e2cc53200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2022-11-27T16:46:40.133 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:40.130+0000 7f4c500ad200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:46:40.220 INFO:tasks.ceph.mgr.x.smithi201.stderr:2022-11-27T16:46:40.218+0000 7f8e2cc53200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2022-11-27T16:46:41.161 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:41.158+0000 7f4c08477700 -1 mgr.server handle_report got status from non-daemon mon.c 2022-11-27T16:46:41.162 INFO:tasks.ceph.mgr.y.smithi145.stderr:2022-11-27T16:46:41.159+0000 7f4c08477700 -1 mgr.server handle_report got status from non-daemon mon.b 2022-11-27T16:46:44.979 DEBUG:teuthology.orchestra.run.smithi145:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2022-11-27T16:46:45.113 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.100+0000 7f6179642700 1 Processor -- start 2022-11-27T16:46:45.113 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.100+0000 7f6179642700 1 -- start start 2022-11-27T16:46:45.114 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.100+0000 7f6179642700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f617415c270 0x7f61740b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.114 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.100+0000 7f6179642700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f61740d0b40 0x7f61740d3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.115 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.100+0000 7f6179642700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f617415c640 con 0x7f61740b1800 2022-11-27T16:46:45.116 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.100+0000 7f6179642700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617406b440 con 0x7f617415c270 2022-11-27T16:46:45.116 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.100+0000 7f6179642700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617406b9a0 con 0x7f61740d0b40 2022-11-27T16:46:45.117 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6172ffd700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f617415c270 0x7f61740b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.117 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f61727fc700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f61740b1800 0x7f61740d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49582/0 (socket says 172.21.15.145:49582) 2022-11-27T16:46:45.117 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f61727fc700 1 -- 172.21.15.145:0/458309829 learned_addr learned my addr 172.21.15.145:0/458309829 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:45.118 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f61737fe700 1 --2- 172.21.15.145:0/458309829 >> v2:172.21.15.201:3300/0 conn(0x7f61740d0b40 0x7f61740d3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.118 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6172ffd700 1 -- 172.21.15.145:0/458309829 >> v2:172.21.15.201:3300/0 conn(0x7f61740d0b40 msgr2=0x7f61740d3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:45.118 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6172ffd700 1 --2- 172.21.15.145:0/458309829 >> v2:172.21.15.201:3300/0 conn(0x7f61740d0b40 0x7f61740d3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.119 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6172ffd700 1 -- 172.21.15.145:0/458309829 >> v1:172.21.15.145:6789/0 conn(0x7f61740b1800 legacy=0x7f61740d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.121 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6172ffd700 1 -- 172.21.15.145:0/458309829 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f617406b6c0 con 0x7f617415c270 2022-11-27T16:46:45.121 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6172ffd700 1 --2- 172.21.15.145:0/458309829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f61740b1d50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61740d2760 tx=0x7f6164009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec42b1186c280ab server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:45.122 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6171ffb700 1 -- 172.21.15.145:0/458309829 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011830 con 0x7f617415c270 2022-11-27T16:46:45.122 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6171ffb700 1 -- 172.21.15.145:0/458309829 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6164011990 con 0x7f617415c270 2022-11-27T16:46:45.122 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.101+0000 7f6171ffb700 1 -- 172.21.15.145:0/458309829 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164011d20 con 0x7f617415c270 2022-11-27T16:46:45.123 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 -- 172.21.15.145:0/458309829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 msgr2=0x7f61740b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.123 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 --2- 172.21.15.145:0/458309829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f61740b1d50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f61740d2760 tx=0x7f6164009b20 comp rx=0 tx=0).stop 2022-11-27T16:46:45.124 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 -- 172.21.15.145:0/458309829 shutdown_connections 2022-11-27T16:46:45.124 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 --2- 172.21.15.145:0/458309829 >> v2:172.21.15.201:3300/0 conn(0x7f61740d0b40 0x7f61740d3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.125 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 --2- 172.21.15.145:0/458309829 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f61740b1d50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.125 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 -- 172.21.15.145:0/458309829 >> 172.21.15.145:0/458309829 conn(0x7f617409bbd0 msgr2=0x7f6174089ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:45.126 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 -- 172.21.15.145:0/458309829 shutdown_connections 2022-11-27T16:46:45.126 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.102+0000 7f6179642700 1 -- 172.21.15.145:0/458309829 wait complete. 2022-11-27T16:46:45.126 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.103+0000 7f6179642700 1 Processor -- start 2022-11-27T16:46:45.127 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.103+0000 7f6179642700 1 -- start start 2022-11-27T16:46:45.127 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6179642700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f61740b1800 0x7f617415f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.128 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6179642700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f61740d0b40 0x7f61740b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.128 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6179642700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f617415aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.128 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6179642700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617415e250 con 0x7f617415c270 2022-11-27T16:46:45.129 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6179642700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617415f0b0 con 0x7f61740d0b40 2022-11-27T16:46:45.129 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6179642700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617415e4a0 con 0x7f61740b1800 2022-11-27T16:46:45.130 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f617415aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.130 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f617415aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54716/0 (socket says 172.21.15.145:54716) 2022-11-27T16:46:45.130 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 -- 172.21.15.145:0/3591914974 learned_addr learned my addr 172.21.15.145:0/3591914974 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:45.131 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61727fc700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f61740d0b40 0x7f61740b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.131 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6172ffd700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f61740b1800 0x7f617415f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.131 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 -- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f61740b1800 msgr2=0x7f617415f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.132 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f61740b1800 0x7f617415f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.132 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 -- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f61740d0b40 msgr2=0x7f61740b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.133 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f61740d0b40 0x7f61740b80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.133 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f61737fe700 1 -- 172.21.15.145:0/3591914974 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61640094a0 con 0x7f617415c270 2022-11-27T16:46:45.134 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.104+0000 7f6172ffd700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f61740b1800 0x7f617415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:45.134 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.105+0000 7f61737fe700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f617415aa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f616c009e30 tx=0x7f616c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:45.134 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.105+0000 7f615b7fe700 1 -- 172.21.15.145:0/3591914974 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c009ca0 con 0x7f617415c270 2022-11-27T16:46:45.134 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.105+0000 7f615b7fe700 1 -- 172.21.15.145:0/3591914974 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616c00f040 con 0x7f617415c270 2022-11-27T16:46:45.135 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.105+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617415e780 con 0x7f617415c270 2022-11-27T16:46:45.135 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.105+0000 7f615b7fe700 1 -- 172.21.15.145:0/3591914974 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616c014480 con 0x7f617415c270 2022-11-27T16:46:45.135 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.105+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617415ed40 con 0x7f617415c270 2022-11-27T16:46:45.135 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.106+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617404e190 con 0x7f617415c270 2022-11-27T16:46:45.136 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.106+0000 7f615b7fe700 1 -- 172.21.15.145:0/3591914974 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f616c0145e0 con 0x7f617415c270 2022-11-27T16:46:45.136 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.107+0000 7f615b7fe700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6154066590 0x7f6154068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.136 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.107+0000 7f6172ffd700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6154066590 0x7f6154068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.136 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.107+0000 7f615b7fe700 1 -- 172.21.15.145:0/3591914974 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f616c084050 con 0x7f617415c270 2022-11-27T16:46:45.136 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.107+0000 7f6172ffd700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6154066590 0x7f6154068a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61640090b0 tx=0x7f616401c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:45.137 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.111+0000 7f615b7fe700 1 -- 172.21.15.145:0/3591914974 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f616c051c10 con 0x7f617415c270 2022-11-27T16:46:45.295 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.293+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61740d2890 con 0x7f617415c270 2022-11-27T16:46:45.296 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.294+0000 7f615b7fe700 1 -- 172.21.15.145:0/3591914974 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12173 (secure 0 0 0) 0x7f616c055230 con 0x7f617415c270 2022-11-27T16:46:45.296 INFO:teuthology.misc.health.smithi145.stdout: 2022-11-27T16:46:45.297 INFO:teuthology.misc.health.smithi145.stdout:{"epoch":16,"fsid":"0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7","created":"2022-11-27T16:46:34.409983+0000","modified":"2022-11-27T16:46:42.889794+0000","last_up_change":"2022-11-27T16:46:39.877265+0000","last_in_change":"2022-11-27T16:46:36.852469+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:46:40.193393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":96845},{"type":"v1","addr":"172.21.15.145:6809","nonce":96845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":96845},{"type":"v1","addr":"172.21.15.145:6811","nonce":96845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6815","nonce":96845},{"type":"v1","addr":"172.21.15.145:6816","nonce":96845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":96845},{"type":"v1","addr":"172.21.15.145:6813","nonce":96845}]},"public_addr":"172.21.15.145:6809/96845","cluster_addr":"172.21.15.145:6811/96845","heartbeat_back_addr":"172.21.15.145:6816/96845","heartbeat_front_addr":"172.21.15.145:6813/96845","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":96844},{"type":"v1","addr":"172.21.15.145:6801","nonce":96844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":96844},{"type":"v1","addr":"172.21.15.145:6803","nonce":96844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":96844},{"type":"v1","addr":"172.21.15.145:6807","nonce":96844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":96844},{"type":"v1","addr":"172.21.15.145:6805","nonce":96844}]},"public_addr":"172.21.15.145:6801/96844","cluster_addr":"172.21.15.145:6803/96844","heartbeat_back_addr":"172.21.15.145:6807/96844","heartbeat_front_addr":"172.21.15.145:6805/96844","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":96848},{"type":"v1","addr":"172.21.15.145:6817","nonce":96848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":96848},{"type":"v1","addr":"172.21.15.145:6819","nonce":96848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":96848},{"type":"v1","addr":"172.21.15.145:6823","nonce":96848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":96848},{"type":"v1","addr":"172.21.15.145:6821","nonce":96848}]},"public_addr":"172.21.15.145:6817/96848","cluster_addr":"172.21.15.145:6819/96848","heartbeat_back_addr":"172.21.15.145:6823/96848","heartbeat_front_addr":"172.21.15.145:6821/96848","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":96860},{"type":"v1","addr":"172.21.15.145:6825","nonce":96860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":96860},{"type":"v1","addr":"172.21.15.145:6827","nonce":96860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":96860},{"type":"v1","addr":"172.21.15.145:6831","nonce":96860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":96860},{"type":"v1","addr":"172.21.15.145:6829","nonce":96860}]},"public_addr":"172.21.15.145:6825/96860","cluster_addr":"172.21.15.145:6827/96860","heartbeat_back_addr":"172.21.15.145:6831/96860","heartbeat_front_addr":"172.21.15.145:6829/96860","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6800","nonce":97125},{"type":"v1","addr":"172.21.15.201:6801","nonce":97125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6802","nonce":97125},{"type":"v1","addr":"172.21.15.201:6803","nonce":97125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6806","nonce":97125},{"type":"v1","addr":"172.21.15.201:6807","nonce":97125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6804","nonce":97125},{"type":"v1","addr":"172.21.15.201:6805","nonce":97125}]},"public_addr":"172.21.15.201:6801/97125","cluster_addr":"172.21.15.201:6803/97125","heartbeat_back_addr":"172.21.15.201:6807/97125","heartbeat_front_addr":"172.21.15.201:6805/97125","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6808","nonce":97126},{"type":"v1","addr":"172.21.15.201:6809","nonce":97126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6810","nonce":97126},{"type":"v1","addr":"172.21.15.201:6811","nonce":97126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6814","nonce":97126},{"type":"v1","addr":"172.21.15.201:6815","nonce":97126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6812","nonce":97126},{"type":"v1","addr":"172.21.15.201:6813","nonce":97126}]},"public_addr":"172.21.15.201:6809/97126","cluster_addr":"172.21.15.201:6811/97126","heartbeat_back_addr":"172.21.15.201:6815/97126","heartbeat_front_addr":"172.21.15.201:6813/97126","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6816","nonce":97127},{"type":"v1","addr":"172.21.15.201:6817","nonce":97127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6818","nonce":97127},{"type":"v1","addr":"172.21.15.201:6819","nonce":97127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6822","nonce":97127},{"type":"v1","addr":"172.21.15.201:6823","nonce":97127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6820","nonce":97127},{"type":"v1","addr":"172.21.15.201:6821","nonce":97127}]},"public_addr":"172.21.15.201:6817/97127","cluster_addr":"172.21.15.201:6819/97127","heartbeat_back_addr":"172.21.15.201:6823/97127","heartbeat_front_addr":"172.21.15.201:6821/97127","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6824","nonce":97224},{"type":"v1","addr":"172.21.15.201:6825","nonce":97224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6826","nonce":97224},{"type":"v1","addr":"172.21.15.201:6827","nonce":97224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6830","nonce":97224},{"type":"v1","addr":"172.21.15.201:6831","nonce":97224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6828","nonce":97224},{"type":"v1","addr":"172.21.15.201:6829","nonce":97224}]},"public_addr":"172.21.15.201:6825/97224","cluster_addr":"172.21.15.201:6827/97224","heartbeat_back_addr":"172.21.15.201:6831/97224","heartbeat_front_addr":"172.21.15.201:6829/97224","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.048726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.994730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.007104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.044350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.979183+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.988046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.037025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.075184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:46:45.298 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6154066590 msgr2=0x7f6154068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.298 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6154066590 0x7f6154068a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61640090b0 tx=0x7f616401c040 comp rx=0 tx=0).stop 2022-11-27T16:46:45.299 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 msgr2=0x7f617415aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.299 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f617415aa00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f616c009e30 tx=0x7f616c00c3a0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.299 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 shutdown_connections 2022-11-27T16:46:45.299 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f617415c270 0x7f617415aa00 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.300 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6154066590 0x7f6154068a40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.300 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f61740b1800 0x7f617415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.300 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 --2- 172.21.15.145:0/3591914974 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f61740d0b40 0x7f61740b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.301 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 >> 172.21.15.145:0/3591914974 conn(0x7f617409bbd0 msgr2=0x7f61740b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:45.301 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 shutdown_connections 2022-11-27T16:46:45.301 INFO:teuthology.misc.health.smithi145.stderr:2022-11-27T16:46:45.296+0000 7f6179642700 1 -- 172.21.15.145:0/3591914974 wait complete. 2022-11-27T16:46:45.309 DEBUG:teuthology.misc:8 of 8 OSDs are up 2022-11-27T16:46:45.309 INFO:tasks.ceph:Creating RBD pool 2022-11-27T16:46:45.310 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd pool create rbd 8 2022-11-27T16:46:45.429 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.427+0000 7fa685a57700 1 Processor -- start 2022-11-27T16:46:45.429 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.427+0000 7fa685a57700 1 -- start start 2022-11-27T16:46:45.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa685a57700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7fa68015c370 0x7fa6800af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa685a57700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa6800d0430 0x7fa6800d08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.432 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa685a57700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6800af970 con 0x7fa6800b0c90 2022-11-27T16:46:45.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa685a57700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6800d0e10 con 0x7fa6800d0430 2022-11-27T16:46:45.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa685a57700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6800afbf0 con 0x7fa68015c370 2022-11-27T16:46:45.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67e7fc700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fa6800d0430 0x7fa6800d08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67f7fe700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fa6800b0c90 0x7fa6800b1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49600/0 (socket says 172.21.15.145:49600) 2022-11-27T16:46:45.433 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67f7fe700 1 -- 172.21.15.145:0/3592529826 learned_addr learned my addr 172.21.15.145:0/3592529826 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:45.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67effd700 1 --2- 172.21.15.145:0/3592529826 >> v2:172.21.15.201:3300/0 conn(0x7fa68015c370 0x7fa6800af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67e7fc700 1 -- 172.21.15.145:0/3592529826 >> v2:172.21.15.201:3300/0 conn(0x7fa68015c370 msgr2=0x7fa6800af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:45.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67e7fc700 1 --2- 172.21.15.145:0/3592529826 >> v2:172.21.15.201:3300/0 conn(0x7fa68015c370 0x7fa6800af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.434 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67e7fc700 1 -- 172.21.15.145:0/3592529826 >> v1:172.21.15.145:6789/0 conn(0x7fa6800b0c90 legacy=0x7fa6800b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.435 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.428+0000 7fa67e7fc700 1 -- 172.21.15.145:0/3592529826 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6800afed0 con 0x7fa6800d0430 2022-11-27T16:46:45.435 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa67e7fc700 1 --2- 172.21.15.145:0/3592529826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800d0430 0x7fa6800d08d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa670009840 tx=0x7fa670009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3546e0972b6b2c3e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:45.435 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa67dffb700 1 -- 172.21.15.145:0/3592529826 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011830 con 0x7fa6800d0430 2022-11-27T16:46:45.435 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa67dffb700 1 -- 172.21.15.145:0/3592529826 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa670011990 con 0x7fa6800d0430 2022-11-27T16:46:45.436 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa67dffb700 1 -- 172.21.15.145:0/3592529826 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670011d20 con 0x7fa6800d0430 2022-11-27T16:46:45.436 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 -- 172.21.15.145:0/3592529826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800d0430 msgr2=0x7fa6800d08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.436 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 --2- 172.21.15.145:0/3592529826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800d0430 0x7fa6800d08d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa670009840 tx=0x7fa670009b50 comp rx=0 tx=0).stop 2022-11-27T16:46:45.436 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 -- 172.21.15.145:0/3592529826 shutdown_connections 2022-11-27T16:46:45.436 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 --2- 172.21.15.145:0/3592529826 >> v2:172.21.15.201:3300/0 conn(0x7fa68015c370 0x7fa6800af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.437 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 --2- 172.21.15.145:0/3592529826 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800d0430 0x7fa6800d08d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.437 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 -- 172.21.15.145:0/3592529826 >> 172.21.15.145:0/3592529826 conn(0x7fa68009bbd0 msgr2=0x7fa6800d2860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:45.437 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 -- 172.21.15.145:0/3592529826 shutdown_connections 2022-11-27T16:46:45.437 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.429+0000 7fa685a57700 1 -- 172.21.15.145:0/3592529826 wait complete. 2022-11-27T16:46:45.441 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.431+0000 7fa685a57700 1 Processor -- start 2022-11-27T16:46:45.441 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.431+0000 7fa685a57700 1 -- start start 2022-11-27T16:46:45.441 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa685a57700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800b0c90 0x7fa68009ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.441 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa685a57700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fa6800d0430 0x7fa68015f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa685a57700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fa68015c370 0x7fa680159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa685a57700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68015f700 con 0x7fa6800b0c90 2022-11-27T16:46:45.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa685a57700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6800a0530 con 0x7fa68015c370 2022-11-27T16:46:45.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa685a57700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68015c870 con 0x7fa6800d0430 2022-11-27T16:46:45.442 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800b0c90 0x7fa68009ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.443 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800b0c90 0x7fa68009ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54726/0 (socket says 172.21.15.145:54726) 2022-11-27T16:46:45.443 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 -- 172.21.15.145:0/3013307594 learned_addr learned my addr 172.21.15.145:0/3013307594 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:45.443 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67f7fe700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fa68015c370 0x7fa680159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.443 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67e7fc700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fa6800d0430 0x7fa68015f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.444 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 -- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fa6800d0430 msgr2=0x7fa68015f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.444 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fa6800d0430 0x7fa68015f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.444 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 -- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fa68015c370 msgr2=0x7fa680159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:45.444 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fa68015c370 0x7fa680159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:45.444 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 -- 172.21.15.145:0/3013307594 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700094a0 con 0x7fa6800b0c90 2022-11-27T16:46:45.445 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67e7fc700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fa6800d0430 0x7fa68015f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:46:45.445 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.432+0000 7fa67effd700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800b0c90 0x7fa68009ff50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa67400e900 tx=0x7fa67400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:45.445 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.433+0000 7fa667fff700 1 -- 172.21.15.145:0/3013307594 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa674009650 con 0x7fa6800b0c90 2022-11-27T16:46:45.445 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.433+0000 7fa667fff700 1 -- 172.21.15.145:0/3013307594 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67400c850 con 0x7fa6800b0c90 2022-11-27T16:46:45.446 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.433+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68015cb50 con 0x7fa6800b0c90 2022-11-27T16:46:45.446 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.433+0000 7fa667fff700 1 -- 172.21.15.145:0/3013307594 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67400cbc0 con 0x7fa6800b0c90 2022-11-27T16:46:45.446 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.433+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68015d110 con 0x7fa6800b0c90 2022-11-27T16:46:45.446 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.433+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68015d480 con 0x7fa6800b0c90 2022-11-27T16:46:45.446 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.434+0000 7fa667fff700 1 -- 172.21.15.145:0/3013307594 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fa674023070 con 0x7fa6800b0c90 2022-11-27T16:46:45.447 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.434+0000 7fa667fff700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fa660066590 0x7fa660068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:45.447 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.434+0000 7fa667fff700 1 -- 172.21.15.145:0/3013307594 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fa6740836d0 con 0x7fa6800b0c90 2022-11-27T16:46:45.447 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.434+0000 7fa67e7fc700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fa660066590 0x7fa660068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:45.447 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.435+0000 7fa67e7fc700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fa660066590 0x7fa660068a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa6800b6eb0 tx=0x7fa670010000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:45.447 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.439+0000 7fa667fff700 1 -- 172.21.15.145:0/3013307594 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fa674051210 con 0x7fa6800b0c90 2022-11-27T16:46:45.619 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:45.617+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7fa68006b740 con 0x7fa6800b0c90 2022-11-27T16:46:46.163 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.161+0000 7fa667fff700 1 -- 172.21.15.145:0/3013307594 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7fa6740170a0 con 0x7fa6800b0c90 2022-11-27T16:46:46.166 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fa660066590 msgr2=0x7fa660068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:46.166 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fa660066590 0x7fa660068a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa6800b6eb0 tx=0x7fa670010000 comp rx=0 tx=0).stop 2022-11-27T16:46:46.166 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800b0c90 msgr2=0x7fa68009ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:46.166 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800b0c90 0x7fa68009ff50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa67400e900 tx=0x7fa67400ecc0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.167 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 shutdown_connections 2022-11-27T16:46:46.167 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fa68015c370 0x7fa680159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.167 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fa6800b0c90 0x7fa68009ff50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.167 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fa660066590 0x7fa660068a40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.168 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 --2- 172.21.15.145:0/3013307594 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fa6800d0430 0x7fa68015f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.168 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 >> 172.21.15.145:0/3013307594 conn(0x7fa68009bbd0 msgr2=0x7fa68006a460 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:46.168 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 shutdown_connections 2022-11-27T16:46:46.168 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.164+0000 7fa685a57700 1 -- 172.21.15.145:0/3013307594 wait complete. 2022-11-27T16:46:46.169 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'rbd' created 2022-11-27T16:46:46.179 DEBUG:teuthology.orchestra.run.smithi145:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2022-11-27T16:46:46.302 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.300+0000 7fd70f199700 1 Processor -- start 2022-11-27T16:46:46.303 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.300+0000 7fd70f199700 1 -- start start 2022-11-27T16:46:46.311 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.300+0000 7fd70f199700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7080cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:46.311 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70f199700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd708160680 0x7fd70815c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:46.311 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70f199700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd70815fc30 con 0x7fd708069380 2022-11-27T16:46:46.312 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70f199700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd708071420 con 0x7fd708160680 2022-11-27T16:46:46.312 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70f199700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd708071700 con 0x7fd708070af0 2022-11-27T16:46:46.313 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70d736700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd708160680 0x7fd70815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:46.313 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70cf35700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fd708069380 0x7fd708069750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49610/0 (socket says 172.21.15.145:49610) 2022-11-27T16:46:46.313 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70cf35700 1 -- 172.21.15.145:0/1692363692 learned_addr learned my addr 172.21.15.145:0/1692363692 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd707fff700 1 --2- 172.21.15.145:0/1692363692 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7080cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70d736700 1 -- 172.21.15.145:0/1692363692 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 msgr2=0x7fd7080cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70d736700 1 --2- 172.21.15.145:0/1692363692 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7080cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70d736700 1 -- 172.21.15.145:0/1692363692 >> v1:172.21.15.145:6789/0 conn(0x7fd708069380 legacy=0x7fd708069750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:46.314 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd7077fe700 1 -- 172.21.15.145:0/1692363692 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1802162204 0 0) 0x7fd70815fc30 con 0x7fd708069380 2022-11-27T16:46:46.315 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70d736700 1 -- 172.21.15.145:0/1692363692 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70815fe80 con 0x7fd708160680 2022-11-27T16:46:46.315 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd707fff700 1 --2- 172.21.15.145:0/1692363692 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7080cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:46:46.315 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd70d736700 1 --2- 172.21.15.145:0/1692363692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708160680 0x7fd70815c590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd700009d90 tx=0x7fd70000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727d9c4ddf4cd63b server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:46.315 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd7077fe700 1 -- 172.21.15.145:0/1692363692 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000e070 con 0x7fd708160680 2022-11-27T16:46:46.316 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd7077fe700 1 -- 172.21.15.145:0/1692363692 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70000bbd0 con 0x7fd708160680 2022-11-27T16:46:46.316 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.301+0000 7fd7077fe700 1 -- 172.21.15.145:0/1692363692 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700012430 con 0x7fd708160680 2022-11-27T16:46:46.316 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 -- 172.21.15.145:0/1692363692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708160680 msgr2=0x7fd70815c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:46.317 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 --2- 172.21.15.145:0/1692363692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708160680 0x7fd70815c590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd700009d90 tx=0x7fd70000b330 comp rx=0 tx=0).stop 2022-11-27T16:46:46.317 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 -- 172.21.15.145:0/1692363692 shutdown_connections 2022-11-27T16:46:46.317 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 --2- 172.21.15.145:0/1692363692 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708160680 0x7fd70815c590 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.317 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 --2- 172.21.15.145:0/1692363692 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7080cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.318 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 -- 172.21.15.145:0/1692363692 >> 172.21.15.145:0/1692363692 conn(0x7fd70806a490 msgr2=0x7fd70806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:46.318 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 -- 172.21.15.145:0/1692363692 shutdown_connections 2022-11-27T16:46:46.318 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.302+0000 7fd70f199700 1 -- 172.21.15.145:0/1692363692 wait complete. 2022-11-27T16:46:46.318 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.303+0000 7fd70f199700 1 Processor -- start 2022-11-27T16:46:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.303+0000 7fd70f199700 1 -- start start 2022-11-27T16:46:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70f199700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708069380 0x7fd7080b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70f199700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7081f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:46.319 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70f199700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd708160680 0x7fd7081f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:46.320 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70f199700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080c8030 con 0x7fd708069380 2022-11-27T16:46:46.320 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70f199700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708151260 con 0x7fd708160680 2022-11-27T16:46:46.320 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70f199700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080c8210 con 0x7fd708070af0 2022-11-27T16:46:46.320 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708069380 0x7fd7080b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:46.321 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708069380 0x7fd7080b17c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54742/0 (socket says 172.21.15.145:54742) 2022-11-27T16:46:46.321 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 -- 172.21.15.145:0/925041784 learned_addr learned my addr 172.21.15.145:0/925041784 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:46.321 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70d736700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd708160680 0x7fd7081f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:46.321 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd707fff700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7081f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:46.322 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 -- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 msgr2=0x7fd7081f4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:46.322 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7081f4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.322 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 -- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd708160680 msgr2=0x7fd7081f5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:46.323 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd708160680 0x7fd7081f5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:46.323 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 -- 172.21.15.145:0/925041784 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009a70 con 0x7fd708069380 2022-11-27T16:46:46.323 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70d736700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd708160680 0x7fd7081f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:46.323 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd707fff700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7081f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:46.324 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd70cf35700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708069380 0x7fd7080b17c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c550 tx=0x7fd6fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:46.324 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.304+0000 7fd7057fa700 1 -- 172.21.15.145:0/925041784 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc004c80 con 0x7fd708069380 2022-11-27T16:46:46.324 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.305+0000 7fd7057fa700 1 -- 172.21.15.145:0/925041784 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc004e00 con 0x7fd708069380 2022-11-27T16:46:46.325 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.305+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7080c84f0 con 0x7fd708069380 2022-11-27T16:46:46.325 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.305+0000 7fd7057fa700 1 -- 172.21.15.145:0/925041784 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc007310 con 0x7fd708069380 2022-11-27T16:46:46.326 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.305+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7080c8ab0 con 0x7fd708069380 2022-11-27T16:46:46.326 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.306+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7080739b0 con 0x7fd708069380 2022-11-27T16:46:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.306+0000 7fd7057fa700 1 -- 172.21.15.145:0/925041784 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fd6fc01a020 con 0x7fd708069380 2022-11-27T16:46:46.327 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.306+0000 7fd7057fa700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6f0066590 0x7fd6f0068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:46.328 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.306+0000 7fd7057fa700 1 -- 172.21.15.145:0/925041784 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7fd6fc082fe0 con 0x7fd708069380 2022-11-27T16:46:46.328 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.306+0000 7fd707fff700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6f0066590 0x7fd6f0068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:46.328 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.307+0000 7fd707fff700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6f0066590 0x7fd6f0068a40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd70815a440 tx=0x7fd6f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:46.329 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.310+0000 7fd7057fa700 1 -- 172.21.15.145:0/925041784 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fd6fc050a50 con 0x7fd708069380 2022-11-27T16:46:46.492 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:46.489+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd7080cc7b0 con 0x7fd708069380 2022-11-27T16:46:47.166 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.164+0000 7fd7057fa700 1 -- 172.21.15.145:0/925041784 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7fd6fc054070 con 0x7fd708069380 2022-11-27T16:46:47.170 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6f0066590 msgr2=0x7fd6f0068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.171 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6f0066590 0x7fd6f0068a40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd70815a440 tx=0x7fd6f4008040 comp rx=0 tx=0).stop 2022-11-27T16:46:47.171 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708069380 msgr2=0x7fd7080b17c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.171 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708069380 0x7fd7080b17c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc00c550 tx=0x7fd6fc00c910 comp rx=0 tx=0).stop 2022-11-27T16:46:47.171 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 shutdown_connections 2022-11-27T16:46:47.172 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd708160680 0x7fd7081f5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.172 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd708069380 0x7fd7080b17c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.173 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6f0066590 0x7fd6f0068a40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.173 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 --2- 172.21.15.145:0/925041784 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd708070af0 0x7fd7081f4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.173 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 >> 172.21.15.145:0/925041784 conn(0x7fd70806a490 msgr2=0x7fd7080c9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:47.174 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 shutdown_connections 2022-11-27T16:46:47.174 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.168+0000 7fd70f199700 1 -- 172.21.15.145:0/925041784 wait complete. 2022-11-27T16:46:47.175 INFO:teuthology.orchestra.run.smithi145.stderr:enabled application 'rbd' on pool 'rbd' 2022-11-27T16:46:47.184 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2022-11-27T16:46:47.185 INFO:tasks.ceph.mds.a:Restarting daemon 2022-11-27T16:46:47.185 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2022-11-27T16:46:47.188 INFO:tasks.ceph.mds.a:Started 2022-11-27T16:46:47.188 INFO:tasks.ceph.mds.c:Restarting daemon 2022-11-27T16:46:47.189 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2022-11-27T16:46:47.191 INFO:tasks.ceph.mds.c:Started 2022-11-27T16:46:47.191 INFO:tasks.ceph.mds.b:Restarting daemon 2022-11-27T16:46:47.192 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2022-11-27T16:46:47.196 INFO:tasks.ceph.mds.b:Started 2022-11-27T16:46:47.196 INFO:tasks.ceph.mds.d:Restarting daemon 2022-11-27T16:46:47.197 DEBUG:teuthology.orchestra.run.smithi201:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2022-11-27T16:46:47.199 INFO:tasks.ceph.mds.d:Started 2022-11-27T16:46:47.200 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2022-11-27T16:46:47.200 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:46:47.273 INFO:tasks.ceph.mds.c.smithi145.stdout:starting mds.c at 2022-11-27T16:46:47.274 INFO:tasks.ceph.mds.a.smithi145.stdout:starting mds.a at 2022-11-27T16:46:47.289 INFO:tasks.ceph.mds.b.smithi201.stdout:starting mds.b at 2022-11-27T16:46:47.290 INFO:tasks.ceph.mds.d.smithi201.stdout:starting mds.d at 2022-11-27T16:46:47.291 INFO:tasks.ceph.mds.d.smithi201.stderr:2022-11-27T16:46:47.285+0000 7f0eaf6ec700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2022-11-27T16:46:47.345 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.334+0000 7f9cd8df4700 1 Processor -- start 2022-11-27T16:46:47.345 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.334+0000 7f9cd8df4700 1 -- start start 2022-11-27T16:46:47.345 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.335+0000 7f9cd8df4700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9cd4070b30 0x7f9cd406ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.345 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.335+0000 7f9cd8df4700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f9cd40cd100 0x7f9cd40cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.346 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.335+0000 7f9cd8df4700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd406e820 con 0x7f9cd4069380 2022-11-27T16:46:47.346 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.335+0000 7f9cd8df4700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd406dfc0 con 0x7f9cd4070b30 2022-11-27T16:46:47.346 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.335+0000 7f9cd8df4700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd4071710 con 0x7f9cd40cd100 2022-11-27T16:46:47.347 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.335+0000 7f9cd1d9b700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9cd4070b30 0x7f9cd406ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.347 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd1d9b700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f9cd4070b30 0x7f9cd406ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54776/0 (socket says 172.21.15.145:54776) 2022-11-27T16:46:47.348 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd1d9b700 1 -- 172.21.15.145:0/374175610 learned_addr learned my addr 172.21.15.145:0/374175610 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:47.348 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd2d9d700 1 --2- 172.21.15.145:0/374175610 >> v2:172.21.15.201:3300/0 conn(0x7f9cd40cd100 0x7f9cd40cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.348 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd1d9b700 1 -- 172.21.15.145:0/374175610 >> v2:172.21.15.201:3300/0 conn(0x7f9cd40cd100 msgr2=0x7f9cd40cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:47.349 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd1d9b700 1 --2- 172.21.15.145:0/374175610 >> v2:172.21.15.201:3300/0 conn(0x7f9cd40cd100 0x7f9cd40cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.349 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd1d9b700 1 -- 172.21.15.145:0/374175610 >> v1:172.21.15.201:6789/0 conn(0x7f9cd4069380 legacy=0x7f9cd4069750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:47.350 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd1d9b700 1 -- 172.21.15.145:0/374175610 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd406e1d0 con 0x7f9cd4070b30 2022-11-27T16:46:47.350 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd1d9b700 1 --2- 172.21.15.145:0/374175610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd4070b30 0x7f9cd406ecb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40c9b10 tx=0x7f9cc4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265cb84c891e5e9 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:47.351 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd159a700 1 -- 172.21.15.145:0/374175610 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4004230 con 0x7f9cd4070b30 2022-11-27T16:46:47.351 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd159a700 1 -- 172.21.15.145:0/374175610 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc4004390 con 0x7f9cd4070b30 2022-11-27T16:46:47.351 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.336+0000 7f9cd159a700 1 -- 172.21.15.145:0/374175610 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40036a0 con 0x7f9cd4070b30 2022-11-27T16:46:47.352 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 -- 172.21.15.145:0/374175610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd4070b30 msgr2=0x7f9cd406ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.352 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/374175610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd4070b30 0x7f9cd406ecb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40c9b10 tx=0x7f9cc4009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.353 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 -- 172.21.15.145:0/374175610 shutdown_connections 2022-11-27T16:46:47.353 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/374175610 >> v2:172.21.15.201:3300/0 conn(0x7f9cd40cd100 0x7f9cd40cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/374175610 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd4070b30 0x7f9cd406ecb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 -- 172.21.15.145:0/374175610 >> 172.21.15.145:0/374175610 conn(0x7f9cd406c6b0 msgr2=0x7f9cd406ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:47.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 -- 172.21.15.145:0/374175610 shutdown_connections 2022-11-27T16:46:47.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.337+0000 7f9cd8df4700 1 -- 172.21.15.145:0/374175610 wait complete. 2022-11-27T16:46:47.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.338+0000 7f9cd8df4700 1 Processor -- start 2022-11-27T16:46:47.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.338+0000 7f9cd8df4700 1 -- start start 2022-11-27T16:46:47.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd8df4700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f9cd4069380 0x7f9cd40c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd8df4700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f9cd4070b30 0x7f9cd40c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.356 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd8df4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd40cd100 0x7f9cd40acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.356 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd8df4700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40c3ff0 con 0x7f9cd40cd100 2022-11-27T16:46:47.356 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd8df4700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40acff0 con 0x7f9cd4069380 2022-11-27T16:46:47.357 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd8df4700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40c3370 con 0x7f9cd4070b30 2022-11-27T16:46:47.357 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd40cd100 0x7f9cd40acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.357 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd40cd100 0x7f9cd40acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54784/0 (socket says 172.21.15.145:54784) 2022-11-27T16:46:47.358 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 -- 172.21.15.145:0/1708365923 learned_addr learned my addr 172.21.15.145:0/1708365923 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:47.358 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd1d9b700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f9cd4070b30 0x7f9cd40c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.359 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd259c700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f9cd4069380 0x7f9cd40c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.359 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 -- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f9cd4070b30 msgr2=0x7f9cd40c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.360 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f9cd4070b30 0x7f9cd40c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.360 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 -- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f9cd4069380 msgr2=0x7f9cd40c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.360 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f9cd4069380 0x7f9cd40c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 -- 172.21.15.145:0/1708365923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4009550 con 0x7f9cd40cd100 2022-11-27T16:46:47.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cd2d9d700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd40cd100 0x7f9cd40acab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00cc30 tx=0x7f9ccc00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:47.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cc2ffd700 1 -- 172.21.15.145:0/1708365923 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc00aa50 con 0x7f9cd40cd100 2022-11-27T16:46:47.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.339+0000 7f9cc2ffd700 1 -- 172.21.15.145:0/1708365923 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc00abb0 con 0x7f9cd40cd100 2022-11-27T16:46:47.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.340+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd40c3650 con 0x7f9cd40cd100 2022-11-27T16:46:47.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.340+0000 7f9cc2ffd700 1 -- 172.21.15.145:0/1708365923 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc003a20 con 0x7f9cd40cd100 2022-11-27T16:46:47.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.340+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd40c3c10 con 0x7f9cd40cd100 2022-11-27T16:46:47.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.340+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd40c4130 con 0x7f9cd40cd100 2022-11-27T16:46:47.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.341+0000 7f9cc2ffd700 1 -- 172.21.15.145:0/1708365923 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f9ccc01f070 con 0x7f9cd40cd100 2022-11-27T16:46:47.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.341+0000 7f9cc2ffd700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f9cb0066590 0x7f9cb0068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.341+0000 7f9cc2ffd700 1 -- 172.21.15.145:0/1708365923 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9ccc083380 con 0x7f9cd40cd100 2022-11-27T16:46:47.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.341+0000 7f9cd259c700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f9cb0066590 0x7f9cb0068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.342+0000 7f9cd259c700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f9cb0066590 0x7f9cb0068a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8006fd0 tx=0x7f9cc80090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:47.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.345+0000 7f9cc2ffd700 1 -- 172.21.15.145:0/1708365923 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f9cd40c4130 con 0x7f9cd40cd100 2022-11-27T16:46:47.527 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.526+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cd40c4130 con 0x7f9cd40cd100 2022-11-27T16:46:47.528 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.526+0000 7f9cc2ffd700 1 -- 172.21.15.145:0/1708365923 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13706 (secure 0 0 0) 0x7f9cd40c4130 con 0x7f9cd40cd100 2022-11-27T16:46:47.529 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:47.529 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":18,"fsid":"0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7","created":"2022-11-27T16:46:34.409983+0000","modified":"2022-11-27T16:46:47.158349+0000","last_up_change":"2022-11-27T16:46:39.877265+0000","last_in_change":"2022-11-27T16:46:36.852469+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:46:40.193393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T16:46:45.635777+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":96845},{"type":"v1","addr":"172.21.15.145:6809","nonce":96845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":96845},{"type":"v1","addr":"172.21.15.145:6811","nonce":96845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6815","nonce":96845},{"type":"v1","addr":"172.21.15.145:6816","nonce":96845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":96845},{"type":"v1","addr":"172.21.15.145:6813","nonce":96845}]},"public_addr":"172.21.15.145:6809/96845","cluster_addr":"172.21.15.145:6811/96845","heartbeat_back_addr":"172.21.15.145:6816/96845","heartbeat_front_addr":"172.21.15.145:6813/96845","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":96844},{"type":"v1","addr":"172.21.15.145:6801","nonce":96844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":96844},{"type":"v1","addr":"172.21.15.145:6803","nonce":96844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":96844},{"type":"v1","addr":"172.21.15.145:6807","nonce":96844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":96844},{"type":"v1","addr":"172.21.15.145:6805","nonce":96844}]},"public_addr":"172.21.15.145:6801/96844","cluster_addr":"172.21.15.145:6803/96844","heartbeat_back_addr":"172.21.15.145:6807/96844","heartbeat_front_addr":"172.21.15.145:6805/96844","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":96848},{"type":"v1","addr":"172.21.15.145:6817","nonce":96848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":96848},{"type":"v1","addr":"172.21.15.145:6819","nonce":96848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":96848},{"type":"v1","addr":"172.21.15.145:6823","nonce":96848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":96848},{"type":"v1","addr":"172.21.15.145:6821","nonce":96848}]},"public_addr":"172.21.15.145:6817/96848","cluster_addr":"172.21.15.145:6819/96848","heartbeat_back_addr":"172.21.15.145:6823/96848","heartbeat_front_addr":"172.21.15.145:6821/96848","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":96860},{"type":"v1","addr":"172.21.15.145:6825","nonce":96860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":96860},{"type":"v1","addr":"172.21.15.145:6827","nonce":96860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":96860},{"type":"v1","addr":"172.21.15.145:6831","nonce":96860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":96860},{"type":"v1","addr":"172.21.15.145:6829","nonce":96860}]},"public_addr":"172.21.15.145:6825/96860","cluster_addr":"172.21.15.145:6827/96860","heartbeat_back_addr":"172.21.15.145:6831/96860","heartbeat_front_addr":"172.21.15.145:6829/96860","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6800","nonce":97125},{"type":"v1","addr":"172.21.15.201:6801","nonce":97125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6802","nonce":97125},{"type":"v1","addr":"172.21.15.201:6803","nonce":97125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6806","nonce":97125},{"type":"v1","addr":"172.21.15.201:6807","nonce":97125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6804","nonce":97125},{"type":"v1","addr":"172.21.15.201:6805","nonce":97125}]},"public_addr":"172.21.15.201:6801/97125","cluster_addr":"172.21.15.201:6803/97125","heartbeat_back_addr":"172.21.15.201:6807/97125","heartbeat_front_addr":"172.21.15.201:6805/97125","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6808","nonce":97126},{"type":"v1","addr":"172.21.15.201:6809","nonce":97126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6810","nonce":97126},{"type":"v1","addr":"172.21.15.201:6811","nonce":97126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6814","nonce":97126},{"type":"v1","addr":"172.21.15.201:6815","nonce":97126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6812","nonce":97126},{"type":"v1","addr":"172.21.15.201:6813","nonce":97126}]},"public_addr":"172.21.15.201:6809/97126","cluster_addr":"172.21.15.201:6811/97126","heartbeat_back_addr":"172.21.15.201:6815/97126","heartbeat_front_addr":"172.21.15.201:6813/97126","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6816","nonce":97127},{"type":"v1","addr":"172.21.15.201:6817","nonce":97127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6818","nonce":97127},{"type":"v1","addr":"172.21.15.201:6819","nonce":97127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6822","nonce":97127},{"type":"v1","addr":"172.21.15.201:6823","nonce":97127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6820","nonce":97127},{"type":"v1","addr":"172.21.15.201:6821","nonce":97127}]},"public_addr":"172.21.15.201:6817/97127","cluster_addr":"172.21.15.201:6819/97127","heartbeat_back_addr":"172.21.15.201:6823/97127","heartbeat_front_addr":"172.21.15.201:6821/97127","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6824","nonce":97224},{"type":"v1","addr":"172.21.15.201:6825","nonce":97224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6826","nonce":97224},{"type":"v1","addr":"172.21.15.201:6827","nonce":97224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6830","nonce":97224},{"type":"v1","addr":"172.21.15.201:6831","nonce":97224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6828","nonce":97224},{"type":"v1","addr":"172.21.15.201:6829","nonce":97224}]},"public_addr":"172.21.15.201:6825/97224","cluster_addr":"172.21.15.201:6827/97224","heartbeat_back_addr":"172.21.15.201:6831/97224","heartbeat_front_addr":"172.21.15.201:6829/97224","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.048726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.994730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.007104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.044350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.979183+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.988046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.037025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.075184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:46:47.531 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.528+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f9cb0066590 msgr2=0x7f9cb0068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.531 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f9cb0066590 0x7f9cb0068a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8006fd0 tx=0x7f9cc80090e0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.532 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd40cd100 msgr2=0x7f9cd40acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.532 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd40cd100 0x7f9cd40acab0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00cc30 tx=0x7f9ccc00cf40 comp rx=0 tx=0).stop 2022-11-27T16:46:47.532 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 shutdown_connections 2022-11-27T16:46:47.532 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f9cd40cd100 0x7f9cd40acab0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.533 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f9cd4069380 0x7f9cd40c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.533 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f9cb0066590 0x7f9cb0068a40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.533 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 --2- 172.21.15.145:0/1708365923 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f9cd4070b30 0x7f9cd40c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.533 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 >> 172.21.15.145:0/1708365923 conn(0x7f9cd406c6b0 msgr2=0x7f9cd40cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:47.534 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 shutdown_connections 2022-11-27T16:46:47.534 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.529+0000 7f9cd8df4700 1 -- 172.21.15.145:0/1708365923 wait complete. 2022-11-27T16:46:47.542 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:46:40.193393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:46:45.635777+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T16:46:47.543 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:46:47.672 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.665+0000 7f6d09ff6700 1 Processor -- start 2022-11-27T16:46:47.673 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d09ff6700 1 -- start start 2022-11-27T16:46:47.673 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d09ff6700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6d040ab8e0 0x7f6d0406a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.674 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d09ff6700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0415d540 con 0x7f6d0415d0d0 2022-11-27T16:46:47.674 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d09ff6700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d04069380 con 0x7f6d040c9350 2022-11-27T16:46:47.675 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d09ff6700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d040695d0 con 0x7f6d040ab8e0 2022-11-27T16:46:47.675 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d03fff700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f6d040ab8e0 0x7f6d0406a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.676 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d02ffd700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f6d0415d0d0 0x7f6d040adf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49666/0 (socket says 172.21.15.145:49666) 2022-11-27T16:46:47.676 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.666+0000 7f6d02ffd700 1 -- 172.21.15.145:0/2801993856 learned_addr learned my addr 172.21.15.145:0/2801993856 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:47.677 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d03fff700 1 -- 172.21.15.145:0/2801993856 >> v1:172.21.15.201:6789/0 conn(0x7f6d040c9350 legacy=0x7f6d040c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.677 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d03fff700 1 -- 172.21.15.145:0/2801993856 >> v1:172.21.15.145:6789/0 conn(0x7f6d0415d0d0 legacy=0x7f6d040adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.677 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d03fff700 1 -- 172.21.15.145:0/2801993856 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d040698b0 con 0x7f6d040ab8e0 2022-11-27T16:46:47.678 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d027fc700 1 -- 172.21.15.145:0/2801993856 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14343317 0 0) 0x7f6d0415d540 con 0x7f6d0415d0d0 2022-11-27T16:46:47.678 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d03fff700 1 --2- 172.21.15.145:0/2801993856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d040ab8e0 0x7f6d0406a390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009d90 tx=0x7f6cfc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f532d0e7e4bc49 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:47.679 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d027fc700 1 -- 172.21.15.145:0/2801993856 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00e070 con 0x7f6d040ab8e0 2022-11-27T16:46:47.679 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d027fc700 1 -- 172.21.15.145:0/2801993856 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00bbd0 con 0x7f6d040ab8e0 2022-11-27T16:46:47.680 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d027fc700 1 -- 172.21.15.145:0/2801993856 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc012430 con 0x7f6d040ab8e0 2022-11-27T16:46:47.680 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d09ff6700 1 -- 172.21.15.145:0/2801993856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d040ab8e0 msgr2=0x7f6d0406a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.681 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.667+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/2801993856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d040ab8e0 0x7f6d0406a390 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc009d90 tx=0x7f6cfc00b330 comp rx=0 tx=0).stop 2022-11-27T16:46:47.681 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.668+0000 7f6d09ff6700 1 -- 172.21.15.145:0/2801993856 shutdown_connections 2022-11-27T16:46:47.682 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.668+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/2801993856 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d040ab8e0 0x7f6d0406a390 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.683 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.668+0000 7f6d09ff6700 1 -- 172.21.15.145:0/2801993856 >> 172.21.15.145:0/2801993856 conn(0x7f6d040cb780 msgr2=0x7f6d040cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:47.683 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.668+0000 7f6d09ff6700 1 -- 172.21.15.145:0/2801993856 shutdown_connections 2022-11-27T16:46:47.683 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.668+0000 7f6d09ff6700 1 -- 172.21.15.145:0/2801993856 wait complete. 2022-11-27T16:46:47.683 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.669+0000 7f6d09ff6700 1 Processor -- start 2022-11-27T16:46:47.687 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.669+0000 7f6d09ff6700 1 -- start start 2022-11-27T16:46:47.687 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f6d040ab8e0 0x7f6d04157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.688 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f6d040c9350 0x7f6d040b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.688 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d0415d0d0 0x7f6d040b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.688 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04156be0 con 0x7f6d0415d0d0 2022-11-27T16:46:47.688 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04155f60 con 0x7f6d040ab8e0 2022-11-27T16:46:47.689 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d04156240 con 0x7f6d040c9350 2022-11-27T16:46:47.689 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d0415d0d0 0x7f6d040b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.689 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d0415d0d0 0x7f6d040b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54798/0 (socket says 172.21.15.145:54798) 2022-11-27T16:46:47.689 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 -- 172.21.15.145:0/1659809495 learned_addr learned my addr 172.21.15.145:0/1659809495 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:47.690 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d037fe700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f6d040ab8e0 0x7f6d04157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.690 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d02ffd700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f6d040c9350 0x7f6d040b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.691 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 -- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f6d040c9350 msgr2=0x7f6d040b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.691 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f6d040c9350 0x7f6d040b2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.692 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 -- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f6d040ab8e0 msgr2=0x7f6d04157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.692 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f6d040ab8e0 0x7f6d04157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.694 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 -- 172.21.15.145:0/1659809495 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc009a70 con 0x7f6d0415d0d0 2022-11-27T16:46:47.694 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d03fff700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d0415d0d0 0x7f6d040b1b90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc014040 tx=0x7f6cfc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:47.694 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6cebfff700 1 -- 172.21.15.145:0/1659809495 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc01b810 con 0x7f6d0415d0d0 2022-11-27T16:46:47.695 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6cebfff700 1 -- 172.21.15.145:0/1659809495 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc020070 con 0x7f6d0415d0d0 2022-11-27T16:46:47.695 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d04156520 con 0x7f6d0415d0d0 2022-11-27T16:46:47.695 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6cebfff700 1 -- 172.21.15.145:0/1659809495 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc01bbf0 con 0x7f6d0415d0d0 2022-11-27T16:46:47.696 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.670+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d041556f0 con 0x7f6d0415d0d0 2022-11-27T16:46:47.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.671+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d040b28a0 con 0x7f6d0415d0d0 2022-11-27T16:46:47.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.672+0000 7f6cebfff700 1 -- 172.21.15.145:0/1659809495 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f6cfc01bd50 con 0x7f6d0415d0d0 2022-11-27T16:46:47.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.672+0000 7f6cebfff700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6ce4066590 0x7f6ce4068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.672+0000 7f6cebfff700 1 -- 172.21.15.145:0/1659809495 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6cfc022070 con 0x7f6d0415d0d0 2022-11-27T16:46:47.698 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.672+0000 7f6d037fe700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6ce4066590 0x7f6ce4068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.698 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.672+0000 7f6d037fe700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6ce4066590 0x7f6ce4068a40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8007fd0 tx=0x7f6cf8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:47.698 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.676+0000 7f6cebfff700 1 -- 172.21.15.145:0/1659809495 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f6d040b28a0 con 0x7f6d0415d0d0 2022-11-27T16:46:47.855 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.852+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d040b28a0 con 0x7f6d0415d0d0 2022-11-27T16:46:47.855 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.853+0000 7f6cebfff700 1 -- 172.21.15.145:0/1659809495 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d040b28a0 con 0x7f6d0415d0d0 2022-11-27T16:46:47.856 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2022-11-27T16:46:47.858 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6ce4066590 msgr2=0x7f6ce4068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.858 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6ce4066590 0x7f6ce4068a40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8007fd0 tx=0x7f6cf8009040 comp rx=0 tx=0).stop 2022-11-27T16:46:47.858 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d0415d0d0 msgr2=0x7f6d040b1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:47.859 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d0415d0d0 0x7f6d040b1b90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc014040 tx=0x7f6cfc004ad0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.859 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 shutdown_connections 2022-11-27T16:46:47.859 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f6d0415d0d0 0x7f6d040b1b90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.859 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f6d040ab8e0 0x7f6d04157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.860 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6ce4066590 0x7f6ce4068a40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.860 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 --2- 172.21.15.145:0/1659809495 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f6d040c9350 0x7f6d040b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.860 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.854+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 >> 172.21.15.145:0/1659809495 conn(0x7f6d040cb780 msgr2=0x7f6d040af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:47.860 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.855+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 shutdown_connections 2022-11-27T16:46:47.861 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.855+0000 7f6d09ff6700 1 -- 172.21.15.145:0/1659809495 wait complete. 2022-11-27T16:46:47.868 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:46:47.991 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.989+0000 7fc640752700 1 Processor -- start 2022-11-27T16:46:47.991 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.989+0000 7fc640752700 1 -- start start 2022-11-27T16:46:47.994 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc640752700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc638156120 0x7fc6381564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.995 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc640752700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7fc6380cd0d0 0x7fc638071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:47.995 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc640752700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc638156a30 con 0x7fc638069380 2022-11-27T16:46:47.995 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc640752700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc63815d300 con 0x7fc638156120 2022-11-27T16:46:47.997 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc640752700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6380b1f90 con 0x7fc6380cd0d0 2022-11-27T16:46:47.997 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc63e4ee700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc638156120 0x7fc6381564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.997 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc63e4ee700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fc638156120 0x7fc6381564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54802/0 (socket says 172.21.15.145:54802) 2022-11-27T16:46:47.997 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.990+0000 7fc63e4ee700 1 -- 172.21.15.145:0/2750314197 learned_addr learned my addr 172.21.15.145:0/2750314197 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:47.998 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63dced700 1 --2- 172.21.15.145:0/2750314197 >> v2:172.21.15.201:3300/0 conn(0x7fc6380cd0d0 0x7fc638071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:47.998 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63e4ee700 1 -- 172.21.15.145:0/2750314197 >> v2:172.21.15.201:3300/0 conn(0x7fc6380cd0d0 msgr2=0x7fc638071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:47.998 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63e4ee700 1 --2- 172.21.15.145:0/2750314197 >> v2:172.21.15.201:3300/0 conn(0x7fc6380cd0d0 0x7fc638071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:47.998 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63e4ee700 1 -- 172.21.15.145:0/2750314197 >> v1:172.21.15.201:6789/0 conn(0x7fc638069380 legacy=0x7fc638069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:47.998 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63e4ee700 1 -- 172.21.15.145:0/2750314197 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63815c680 con 0x7fc638156120 2022-11-27T16:46:47.999 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63e4ee700 1 --2- 172.21.15.145:0/2750314197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc638156120 0x7fc6381564f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d05a0 tx=0x7fc628009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bf202ffebbb030 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63d4ec700 1 -- 172.21.15.145:0/2750314197 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628005e30 con 0x7fc638156120 2022-11-27T16:46:48.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63d4ec700 1 -- 172.21.15.145:0/2750314197 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc628006030 con 0x7fc638156120 2022-11-27T16:46:48.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc63d4ec700 1 -- 172.21.15.145:0/2750314197 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6280063e0 con 0x7fc638156120 2022-11-27T16:46:48.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc640752700 1 -- 172.21.15.145:0/2750314197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc638156120 msgr2=0x7fc6381564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.001 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.991+0000 7fc640752700 1 --2- 172.21.15.145:0/2750314197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc638156120 0x7fc6381564f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d05a0 tx=0x7fc628009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.001 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.992+0000 7fc640752700 1 -- 172.21.15.145:0/2750314197 shutdown_connections 2022-11-27T16:46:48.001 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.992+0000 7fc640752700 1 --2- 172.21.15.145:0/2750314197 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc638156120 0x7fc6381564f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.002 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.992+0000 7fc640752700 1 --2- 172.21.15.145:0/2750314197 >> v2:172.21.15.201:3300/0 conn(0x7fc6380cd0d0 0x7fc638071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.002 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.992+0000 7fc640752700 1 -- 172.21.15.145:0/2750314197 >> 172.21.15.145:0/2750314197 conn(0x7fc63806a490 msgr2=0x7fc63806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:48.002 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.992+0000 7fc640752700 1 -- 172.21.15.145:0/2750314197 shutdown_connections 2022-11-27T16:46:48.003 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.992+0000 7fc640752700 1 -- 172.21.15.145:0/2750314197 wait complete. 2022-11-27T16:46:48.003 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.993+0000 7fc640752700 1 Processor -- start 2022-11-27T16:46:48.003 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.993+0000 7fc640752700 1 -- start start 2022-11-27T16:46:48.007 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc638069380 0x7fc6380c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.007 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6380cd0d0 0x7fc6380b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.007 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc638156120 0x7fc6380c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.007 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380b4c20 con 0x7fc6380cd0d0 2022-11-27T16:46:48.008 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380b4f00 con 0x7fc638069380 2022-11-27T16:46:48.008 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6380c7830 con 0x7fc638156120 2022-11-27T16:46:48.008 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6380cd0d0 0x7fc6380b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.008 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6380cd0d0 0x7fc6380b4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54810/0 (socket says 172.21.15.145:54810) 2022-11-27T16:46:48.009 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 -- 172.21.15.145:0/2775444865 learned_addr learned my addr 172.21.15.145:0/2775444865 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:48.009 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63e4ee700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc638069380 0x7fc6380c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.009 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63ecef700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc638156120 0x7fc6380c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.009 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 -- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc638156120 msgr2=0x7fc6380c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc638156120 0x7fc6380c4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 -- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc638069380 msgr2=0x7fc6380c8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc638069380 0x7fc6380c8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 -- 172.21.15.145:0/2775444865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc628009500 con 0x7fc6380cd0d0 2022-11-27T16:46:48.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc63dced700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6380cd0d0 0x7fc6380b4340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc63400e810 tx=0x7fc63400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc626ffd700 1 -- 172.21.15.145:0/2775444865 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63400c070 con 0x7fc6380cd0d0 2022-11-27T16:46:48.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc626ffd700 1 -- 172.21.15.145:0/2775444865 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc634013610 con 0x7fc6380cd0d0 2022-11-27T16:46:48.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6380c7b10 con 0x7fc6380cd0d0 2022-11-27T16:46:48.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc626ffd700 1 -- 172.21.15.145:0/2775444865 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6340139c0 con 0x7fc6380cd0d0 2022-11-27T16:46:48.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.994+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc638155150 con 0x7fc6380cd0d0 2022-11-27T16:46:48.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.995+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6381553d0 con 0x7fc6380cd0d0 2022-11-27T16:46:48.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.996+0000 7fc626ffd700 1 -- 172.21.15.145:0/2775444865 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fc634023070 con 0x7fc6380cd0d0 2022-11-27T16:46:48.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.996+0000 7fc626ffd700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fc618066590 0x7fc618068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.996+0000 7fc626ffd700 1 -- 172.21.15.145:0/2775444865 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc6340831d0 con 0x7fc6380cd0d0 2022-11-27T16:46:48.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.996+0000 7fc63e4ee700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fc618066590 0x7fc618068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:47.996+0000 7fc63e4ee700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fc618066590 0x7fc618068a40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d05a0 tx=0x7fc628008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.000+0000 7fc626ffd700 1 -- 172.21.15.145:0/2775444865 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fc634050bb0 con 0x7fc6380cd0d0 2022-11-27T16:46:48.180 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.178+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6380739b0 con 0x7fc6380cd0d0 2022-11-27T16:46:48.180 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.178+0000 7fc626ffd700 1 -- 172.21.15.145:0/2775444865 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc63400f040 con 0x7fc6380cd0d0 2022-11-27T16:46:48.181 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 8 2022-11-27T16:46:48.182 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fc618066590 msgr2=0x7fc618068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.183 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fc618066590 0x7fc618068a40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc6380d05a0 tx=0x7fc628008070 comp rx=0 tx=0).stop 2022-11-27T16:46:48.183 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6380cd0d0 msgr2=0x7fc6380b4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.183 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6380cd0d0 0x7fc6380b4340 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc63400e810 tx=0x7fc63400eb20 comp rx=0 tx=0).stop 2022-11-27T16:46:48.184 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 shutdown_connections 2022-11-27T16:46:48.184 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fc618066590 0x7fc618068a40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.184 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fc638156120 0x7fc6380c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.184 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fc638069380 0x7fc6380c8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.185 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 --2- 172.21.15.145:0/2775444865 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fc6380cd0d0 0x7fc6380b4340 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.185 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.180+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 >> 172.21.15.145:0/2775444865 conn(0x7fc63806a490 msgr2=0x7fc638069750 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:48.185 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.181+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 shutdown_connections 2022-11-27T16:46:48.185 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.181+0000 7fc640752700 1 -- 172.21.15.145:0/2775444865 wait complete. 2022-11-27T16:46:48.195 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2022-11-27T16:46:48.318 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.316+0000 7fd9740b0700 1 Processor -- start 2022-11-27T16:46:48.319 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.317+0000 7fd9740b0700 1 -- start start 2022-11-27T16:46:48.325 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.317+0000 7fd9740b0700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd96c159fd0 0x7fd96c0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.325 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.317+0000 7fd9740b0700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c06a5a0 0x7fd96c06be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.326 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.317+0000 7fd9740b0700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7fd96c06c3b0 0x7fd96c069380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.326 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.317+0000 7fd9740b0700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c06c830 con 0x7fd96c159fd0 2022-11-27T16:46:48.326 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.317+0000 7fd9740b0700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd96c15a3a0 con 0x7fd96c06c3b0 2022-11-27T16:46:48.326 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.317+0000 7fd9740b0700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c06ab60 con 0x7fd96c06a5a0 2022-11-27T16:46:48.327 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd96c159fd0 0x7fd96c0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.327 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fd96c159fd0 0x7fd96c0c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54818/0 (socket says 172.21.15.145:54818) 2022-11-27T16:46:48.327 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 -- 172.21.15.145:0/998445096 learned_addr learned my addr 172.21.15.145:0/998445096 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:48.327 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd97264d700 1 --2- 172.21.15.145:0/998445096 >> v2:172.21.15.201:3300/0 conn(0x7fd96c06c3b0 0x7fd96c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.328 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd97164b700 1 --2- 172.21.15.145:0/998445096 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c06a5a0 0x7fd96c06be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.328 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 -- 172.21.15.145:0/998445096 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c06a5a0 msgr2=0x7fd96c06be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:48.328 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 --2- 172.21.15.145:0/998445096 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c06a5a0 0x7fd96c06be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.328 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 -- 172.21.15.145:0/998445096 >> v2:172.21.15.201:3300/0 conn(0x7fd96c06c3b0 msgr2=0x7fd96c069380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:48.329 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 --2- 172.21.15.145:0/998445096 >> v2:172.21.15.201:3300/0 conn(0x7fd96c06c3b0 0x7fd96c069380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.329 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 -- 172.21.15.145:0/998445096 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd96c15da70 con 0x7fd96c159fd0 2022-11-27T16:46:48.329 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd971e4c700 1 --2- 172.21.15.145:0/998445096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c159fd0 0x7fd96c0c9270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0cbed0 tx=0x7fd95c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6daf4b34e8299513 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.329 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd970e4a700 1 -- 172.21.15.145:0/998445096 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00b870 con 0x7fd96c159fd0 2022-11-27T16:46:48.330 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd970e4a700 1 -- 172.21.15.145:0/998445096 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c00b9d0 con 0x7fd96c159fd0 2022-11-27T16:46:48.330 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.318+0000 7fd970e4a700 1 -- 172.21.15.145:0/998445096 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00bd60 con 0x7fd96c159fd0 2022-11-27T16:46:48.330 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 -- 172.21.15.145:0/998445096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c159fd0 msgr2=0x7fd96c0c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.330 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 --2- 172.21.15.145:0/998445096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c159fd0 0x7fd96c0c9270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0cbed0 tx=0x7fd95c009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.331 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 -- 172.21.15.145:0/998445096 shutdown_connections 2022-11-27T16:46:48.331 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 --2- 172.21.15.145:0/998445096 >> v2:172.21.15.201:3300/0 conn(0x7fd96c06c3b0 0x7fd96c069380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.331 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 --2- 172.21.15.145:0/998445096 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c159fd0 0x7fd96c0c9270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.331 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 --2- 172.21.15.145:0/998445096 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c06a5a0 0x7fd96c06be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.332 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 -- 172.21.15.145:0/998445096 >> 172.21.15.145:0/998445096 conn(0x7fd96c0cf940 msgr2=0x7fd96c0cb730 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:48.332 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 -- 172.21.15.145:0/998445096 shutdown_connections 2022-11-27T16:46:48.332 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.319+0000 7fd9740b0700 1 -- 172.21.15.145:0/998445096 wait complete. 2022-11-27T16:46:48.332 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.320+0000 7fd9740b0700 1 Processor -- start 2022-11-27T16:46:48.332 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.320+0000 7fd9740b0700 1 -- start start 2022-11-27T16:46:48.333 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c06a5a0 0x7fd96c1e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.333 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd96c06c3b0 0x7fd96c1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.333 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c159fd0 0x7fd96c1ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.333 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0af8c0 con 0x7fd96c06a5a0 2022-11-27T16:46:48.334 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c1e39f0 con 0x7fd96c06c3b0 2022-11-27T16:46:48.334 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd96c0afb10 con 0x7fd96c159fd0 2022-11-27T16:46:48.334 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c06a5a0 0x7fd96c1e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.334 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c06a5a0 0x7fd96c1e34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54824/0 (socket says 172.21.15.145:54824) 2022-11-27T16:46:48.335 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 -- 172.21.15.145:0/1944216545 learned_addr learned my addr 172.21.15.145:0/1944216545 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:48.335 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd97164b700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd96c06c3b0 0x7fd96c1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.335 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd97264d700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c159fd0 0x7fd96c1ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.335 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 -- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c159fd0 msgr2=0x7fd96c1ef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.336 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c159fd0 0x7fd96c1ef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.336 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 -- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd96c06c3b0 msgr2=0x7fd96c1569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.336 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd96c06c3b0 0x7fd96c1569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.336 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 -- 172.21.15.145:0/1944216545 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c009550 con 0x7fd96c06a5a0 2022-11-27T16:46:48.337 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd97264d700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c159fd0 0x7fd96c1ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:48.337 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd971e4c700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c06a5a0 0x7fd96c1e34b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd95c014040 tx=0x7fd95c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.337 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd95a7fc700 1 -- 172.21.15.145:0/1944216545 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c004310 con 0x7fd96c06a5a0 2022-11-27T16:46:48.337 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd95a7fc700 1 -- 172.21.15.145:0/1944216545 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95c01e070 con 0x7fd96c06a5a0 2022-11-27T16:46:48.338 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96c0afdf0 con 0x7fd96c06a5a0 2022-11-27T16:46:48.338 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd95a7fc700 1 -- 172.21.15.145:0/1944216545 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95c0036a0 con 0x7fd96c06a5a0 2022-11-27T16:46:48.338 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.321+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c0b03b0 con 0x7fd96c06a5a0 2022-11-27T16:46:48.338 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.322+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd96c0739b0 con 0x7fd96c06a5a0 2022-11-27T16:46:48.339 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.323+0000 7fd95a7fc700 1 -- 172.21.15.145:0/1944216545 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fd95c029020 con 0x7fd96c06a5a0 2022-11-27T16:46:48.339 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.323+0000 7fd95a7fc700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd94c066590 0x7fd94c068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.339 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.323+0000 7fd95a7fc700 1 -- 172.21.15.145:0/1944216545 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd95c084690 con 0x7fd96c06a5a0 2022-11-27T16:46:48.339 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.323+0000 7fd97164b700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd94c066590 0x7fd94c068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.340 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.323+0000 7fd97164b700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd94c066590 0x7fd94c068a40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd96c1f2cc0 tx=0x7fd968008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.340 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.327+0000 7fd95a7fc700 1 -- 172.21.15.145:0/1944216545 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fd95c0520f0 con 0x7fd96c06a5a0 2022-11-27T16:46:48.551 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.549+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd96c15dd50 con 0x7fd96c06a5a0 2022-11-27T16:46:48.552 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.549+0000 7fd95a7fc700 1 -- 172.21.15.145:0/1944216545 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3530 (secure 0 0 0) 0x7fd95c015de0 con 0x7fd96c06a5a0 2022-11-27T16:46:48.552 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:48.553 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4226,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.201:6835/378724109","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6833","nonce":378724109},{"type":"v1","addr":"172.21.15.201:6835","nonce":378724109}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4267,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6837/3714815902","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6834","nonce":3714815902},{"type":"v1","addr":"172.21.15.145:6837","nonce":3714815902}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4276,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.201:6834/272879675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6832","nonce":272879675},{"type":"v1","addr":"172.21.15.201:6834","nonce":272879675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.145:6836/3343174545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6835","nonce":3343174545},{"type":"v1","addr":"172.21.15.145:6836","nonce":3343174545}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2022-11-27T16:46:48.554 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd94c066590 msgr2=0x7fd94c068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.554 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd94c066590 0x7fd94c068a40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd96c1f2cc0 tx=0x7fd968008040 comp rx=0 tx=0).stop 2022-11-27T16:46:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c06a5a0 msgr2=0x7fd96c1e34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c06a5a0 0x7fd96c1e34b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd95c014040 tx=0x7fd95c004750 comp rx=0 tx=0).stop 2022-11-27T16:46:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 shutdown_connections 2022-11-27T16:46:48.555 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd94c066590 0x7fd94c068a40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.556 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd96c159fd0 0x7fd96c1ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.556 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd96c06a5a0 0x7fd96c1e34b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.556 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.551+0000 7fd9740b0700 1 --2- 172.21.15.145:0/1944216545 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd96c06c3b0 0x7fd96c1569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.556 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.552+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 >> 172.21.15.145:0/1944216545 conn(0x7fd96c0cf940 msgr2=0x7fd96c0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:48.556 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.552+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 shutdown_connections 2022-11-27T16:46:48.557 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.552+0000 7fd9740b0700 1 -- 172.21.15.145:0/1944216545 wait complete. 2022-11-27T16:46:48.557 INFO:teuthology.orchestra.run.smithi145.stderr:dumped fsmap epoch 3 2022-11-27T16:46:48.567 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:46:48.687 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.685+0000 7f693891a700 1 Processor -- start 2022-11-27T16:46:48.687 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.685+0000 7f693891a700 1 -- start start 2022-11-27T16:46:48.694 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693891a700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69340ad2f0 0x7f69340d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.694 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693891a700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f693415a140 0x7f69340af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.695 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693891a700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340b0630 0x7f69340b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.695 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693891a700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340d1640 con 0x7f69340ad2f0 2022-11-27T16:46:48.696 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693891a700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69340afb90 con 0x7f693415a140 2022-11-27T16:46:48.696 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693891a700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69340af060 con 0x7f69340b0630 2022-11-27T16:46:48.696 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69340ad2f0 0x7f69340d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f69340ad2f0 0x7f69340d00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54832/0 (socket says 172.21.15.145:54832) 2022-11-27T16:46:48.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 -- 172.21.15.145:0/1873361767 learned_addr learned my addr 172.21.15.145:0/1873361767 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:48.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f6932d9d700 1 --2- 172.21.15.145:0/1873361767 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340b0630 0x7f69340b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.698 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f6931d9b700 1 --2- 172.21.15.145:0/1873361767 >> v2:172.21.15.201:3300/0 conn(0x7f693415a140 0x7f69340af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.698 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 -- 172.21.15.145:0/1873361767 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340b0630 msgr2=0x7f69340b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:48.699 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 --2- 172.21.15.145:0/1873361767 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340b0630 0x7f69340b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.699 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 -- 172.21.15.145:0/1873361767 >> v2:172.21.15.201:3300/0 conn(0x7f693415a140 msgr2=0x7f69340af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:48.700 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 --2- 172.21.15.145:0/1873361767 >> v2:172.21.15.201:3300/0 conn(0x7f693415a140 0x7f69340af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.700 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 -- 172.21.15.145:0/1873361767 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340af860 con 0x7f69340ad2f0 2022-11-27T16:46:48.701 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f6931d9b700 1 --2- 172.21.15.145:0/1873361767 >> v2:172.21.15.201:3300/0 conn(0x7f693415a140 0x7f69340af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:46:48.701 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693259c700 1 --2- 172.21.15.145:0/1873361767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340ad2f0 0x7f69340d00c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69340b3280 tx=0x7f6924009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b77f5067b0f53f7a server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.702 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693159a700 1 -- 172.21.15.145:0/1873361767 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6924005e30 con 0x7f69340ad2f0 2022-11-27T16:46:48.702 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693159a700 1 -- 172.21.15.145:0/1873361767 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6924004030 con 0x7f69340ad2f0 2022-11-27T16:46:48.702 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.686+0000 7f693159a700 1 -- 172.21.15.145:0/1873361767 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240043e0 con 0x7f69340ad2f0 2022-11-27T16:46:48.703 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 -- 172.21.15.145:0/1873361767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340ad2f0 msgr2=0x7f69340d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.703 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 --2- 172.21.15.145:0/1873361767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340ad2f0 0x7f69340d00c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f69340b3280 tx=0x7f6924009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.704 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 -- 172.21.15.145:0/1873361767 shutdown_connections 2022-11-27T16:46:48.704 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 --2- 172.21.15.145:0/1873361767 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340b0630 0x7f69340b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.705 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 --2- 172.21.15.145:0/1873361767 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340ad2f0 0x7f69340d00c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.705 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 --2- 172.21.15.145:0/1873361767 >> v2:172.21.15.201:3300/0 conn(0x7f693415a140 0x7f69340af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.706 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 -- 172.21.15.145:0/1873361767 >> 172.21.15.145:0/1873361767 conn(0x7f6934099a00 msgr2=0x7f69340d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:48.706 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 -- 172.21.15.145:0/1873361767 shutdown_connections 2022-11-27T16:46:48.707 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.687+0000 7f693891a700 1 -- 172.21.15.145:0/1873361767 wait complete. 2022-11-27T16:46:48.707 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.688+0000 7f693891a700 1 Processor -- start 2022-11-27T16:46:48.707 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.688+0000 7f693891a700 1 -- start start 2022-11-27T16:46:48.708 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f693891a700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340ad2f0 0x7f693414e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.708 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f693891a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340b0630 0x7f6934151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.709 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f693891a700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f693415a140 0x7f69340b58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.709 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f693891a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934152450 con 0x7f69340b0630 2022-11-27T16:46:48.709 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f693891a700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6934152730 con 0x7f693415a140 2022-11-27T16:46:48.710 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f693891a700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6934152a10 con 0x7f69340ad2f0 2022-11-27T16:46:48.710 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f6931d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340b0630 0x7f6934151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.711 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f6931d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340b0630 0x7f6934151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54840/0 (socket says 172.21.15.145:54840) 2022-11-27T16:46:48.711 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f6931d9b700 1 -- 172.21.15.145:0/4264555800 learned_addr learned my addr 172.21.15.145:0/4264555800 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:48.712 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f6932d9d700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f693415a140 0x7f69340b58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.712 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f693259c700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340ad2f0 0x7f693414e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.713 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f6931d9b700 1 -- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340ad2f0 msgr2=0x7f693414e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.713 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f6931d9b700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340ad2f0 0x7f693414e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.714 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.689+0000 7f6931d9b700 1 -- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f693415a140 msgr2=0x7f69340b58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.714 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f6931d9b700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f693415a140 0x7f69340b58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.715 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f6931d9b700 1 -- 172.21.15.145:0/4264555800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009500 con 0x7f69340b0630 2022-11-27T16:46:48.715 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f693259c700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340ad2f0 0x7f693414e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:48.715 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f6931d9b700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340b0630 0x7f6934151ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f692800c7f0 tx=0x7f692800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.716 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f6922ffd700 1 -- 172.21.15.145:0/4264555800 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928011800 con 0x7f69340b0630 2022-11-27T16:46:48.716 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f6922ffd700 1 -- 172.21.15.145:0/4264555800 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6928011960 con 0x7f69340b0630 2022-11-27T16:46:48.717 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6934152cf0 con 0x7f69340b0630 2022-11-27T16:46:48.717 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f6922ffd700 1 -- 172.21.15.145:0/4264555800 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928011cf0 con 0x7f69340b0630 2022-11-27T16:46:48.717 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.690+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6934152020 con 0x7f69340b0630 2022-11-27T16:46:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.691+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69340739b0 con 0x7f69340b0630 2022-11-27T16:46:48.718 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.691+0000 7f6922ffd700 1 -- 172.21.15.145:0/4264555800 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f6928021070 con 0x7f69340b0630 2022-11-27T16:46:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.691+0000 7f6922ffd700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6910066590 0x7f6910068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.692+0000 7f6922ffd700 1 -- 172.21.15.145:0/4264555800 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6928083250 con 0x7f69340b0630 2022-11-27T16:46:48.719 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.692+0000 7f693259c700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6910066590 0x7f6910068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:48.720 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.692+0000 7f693259c700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6910066590 0x7f6910068a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f69340b3280 tx=0x7f6924008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:48.721 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.696+0000 7f6922ffd700 1 -- 172.21.15.145:0/4264555800 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f6928050c30 con 0x7f69340b0630 2022-11-27T16:46:48.876 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.874+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f693406f360 con 0x7f69340b0630 2022-11-27T16:46:48.877 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.875+0000 7f6922ffd700 1 -- 172.21.15.145:0/4264555800 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13693 (secure 0 0 0) 0x7f6928054250 con 0x7f69340b0630 2022-11-27T16:46:48.878 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:48.878 INFO:teuthology.orchestra.run.smithi145.stdout:{"epoch":19,"fsid":"0b21400c-efc0-4dbe-9ae0-bcc1b13be5a7","created":"2022-11-27T16:46:34.409983+0000","modified":"2022-11-27T16:46:48.163910+0000","last_up_change":"2022-11-27T16:46:39.877265+0000","last_in_change":"2022-11-27T16:46:36.852469+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":".mgr","create_time":"2022-11-27T16:46:40.193393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2022-11-27T16:46:45.635777+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6808","nonce":96845},{"type":"v1","addr":"172.21.15.145:6809","nonce":96845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6810","nonce":96845},{"type":"v1","addr":"172.21.15.145:6811","nonce":96845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6815","nonce":96845},{"type":"v1","addr":"172.21.15.145:6816","nonce":96845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6812","nonce":96845},{"type":"v1","addr":"172.21.15.145:6813","nonce":96845}]},"public_addr":"172.21.15.145:6809/96845","cluster_addr":"172.21.15.145:6811/96845","heartbeat_back_addr":"172.21.15.145:6816/96845","heartbeat_front_addr":"172.21.15.145:6813/96845","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6800","nonce":96844},{"type":"v1","addr":"172.21.15.145:6801","nonce":96844}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6802","nonce":96844},{"type":"v1","addr":"172.21.15.145:6803","nonce":96844}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6806","nonce":96844},{"type":"v1","addr":"172.21.15.145:6807","nonce":96844}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6804","nonce":96844},{"type":"v1","addr":"172.21.15.145:6805","nonce":96844}]},"public_addr":"172.21.15.145:6801/96844","cluster_addr":"172.21.15.145:6803/96844","heartbeat_back_addr":"172.21.15.145:6807/96844","heartbeat_front_addr":"172.21.15.145:6805/96844","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6814","nonce":96848},{"type":"v1","addr":"172.21.15.145:6817","nonce":96848}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6818","nonce":96848},{"type":"v1","addr":"172.21.15.145:6819","nonce":96848}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6822","nonce":96848},{"type":"v1","addr":"172.21.15.145:6823","nonce":96848}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6820","nonce":96848},{"type":"v1","addr":"172.21.15.145:6821","nonce":96848}]},"public_addr":"172.21.15.145:6817/96848","cluster_addr":"172.21.15.145:6819/96848","heartbeat_back_addr":"172.21.15.145:6823/96848","heartbeat_front_addr":"172.21.15.145:6821/96848","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6824","nonce":96860},{"type":"v1","addr":"172.21.15.145:6825","nonce":96860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6826","nonce":96860},{"type":"v1","addr":"172.21.15.145:6827","nonce":96860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6830","nonce":96860},{"type":"v1","addr":"172.21.15.145:6831","nonce":96860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.145:6828","nonce":96860},{"type":"v1","addr":"172.21.15.145:6829","nonce":96860}]},"public_addr":"172.21.15.145:6825/96860","cluster_addr":"172.21.15.145:6827/96860","heartbeat_back_addr":"172.21.15.145:6831/96860","heartbeat_front_addr":"172.21.15.145:6829/96860","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6800","nonce":97125},{"type":"v1","addr":"172.21.15.201:6801","nonce":97125}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6802","nonce":97125},{"type":"v1","addr":"172.21.15.201:6803","nonce":97125}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6806","nonce":97125},{"type":"v1","addr":"172.21.15.201:6807","nonce":97125}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6804","nonce":97125},{"type":"v1","addr":"172.21.15.201:6805","nonce":97125}]},"public_addr":"172.21.15.201:6801/97125","cluster_addr":"172.21.15.201:6803/97125","heartbeat_back_addr":"172.21.15.201:6807/97125","heartbeat_front_addr":"172.21.15.201:6805/97125","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6808","nonce":97126},{"type":"v1","addr":"172.21.15.201:6809","nonce":97126}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6810","nonce":97126},{"type":"v1","addr":"172.21.15.201:6811","nonce":97126}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6814","nonce":97126},{"type":"v1","addr":"172.21.15.201:6815","nonce":97126}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6812","nonce":97126},{"type":"v1","addr":"172.21.15.201:6813","nonce":97126}]},"public_addr":"172.21.15.201:6809/97126","cluster_addr":"172.21.15.201:6811/97126","heartbeat_back_addr":"172.21.15.201:6815/97126","heartbeat_front_addr":"172.21.15.201:6813/97126","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6816","nonce":97127},{"type":"v1","addr":"172.21.15.201:6817","nonce":97127}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6818","nonce":97127},{"type":"v1","addr":"172.21.15.201:6819","nonce":97127}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6822","nonce":97127},{"type":"v1","addr":"172.21.15.201:6823","nonce":97127}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6820","nonce":97127},{"type":"v1","addr":"172.21.15.201:6821","nonce":97127}]},"public_addr":"172.21.15.201:6817/97127","cluster_addr":"172.21.15.201:6819/97127","heartbeat_back_addr":"172.21.15.201:6823/97127","heartbeat_front_addr":"172.21.15.201:6821/97127","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6824","nonce":97224},{"type":"v1","addr":"172.21.15.201:6825","nonce":97224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6826","nonce":97224},{"type":"v1","addr":"172.21.15.201:6827","nonce":97224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6830","nonce":97224},{"type":"v1","addr":"172.21.15.201:6831","nonce":97224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.201:6828","nonce":97224},{"type":"v1","addr":"172.21.15.201:6829","nonce":97224}]},"public_addr":"172.21.15.201:6825/97224","cluster_addr":"172.21.15.201:6827/97224","heartbeat_back_addr":"172.21.15.201:6831/97224","heartbeat_front_addr":"172.21.15.201:6829/97224","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.048726+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.994730+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.007104+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.044350+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.979183+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:37.988046+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.037025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2022-11-27T16:46:38.075184+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2022-11-27T16:46:48.880 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6910066590 msgr2=0x7f6910068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.880 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6910066590 0x7f6910068a40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f69340b3280 tx=0x7f6924008070 comp rx=0 tx=0).stop 2022-11-27T16:46:48.881 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340b0630 msgr2=0x7f6934151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:48.881 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340b0630 0x7f6934151ae0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f692800c7f0 tx=0x7f692800cb00 comp rx=0 tx=0).stop 2022-11-27T16:46:48.881 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 shutdown_connections 2022-11-27T16:46:48.882 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f693415a140 0x7f69340b58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.882 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f6910066590 0x7f6910068a40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.883 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f69340ad2f0 0x7f693414e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.883 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 --2- 172.21.15.145:0/4264555800 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f69340b0630 0x7f6934151ae0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:48.884 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 >> 172.21.15.145:0/4264555800 conn(0x7f6934099a00 msgr2=0x7f693406d920 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:48.884 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 shutdown_connections 2022-11-27T16:46:48.884 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:48.877+0000 7f693891a700 1 -- 172.21.15.145:0/4264555800 wait complete. 2022-11-27T16:46:48.892 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2022-11-27T16:46:40.193393+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2022-11-27T16:46:45.635777+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2022-11-27T16:46:48.893 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2022-11-27T16:46:49.018 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.013+0000 7f1f071a1700 1 Processor -- start 2022-11-27T16:46:49.018 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.013+0000 7f1f071a1700 1 -- start start 2022-11-27T16:46:49.018 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1f071a1700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1f000b0ea0 0x7f1f000ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.019 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1f071a1700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f1f000b1ca0 0x7f1f000b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.019 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1f071a1700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f000d1570 con 0x7f1f000b0ad0 2022-11-27T16:46:49.025 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1f071a1700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f000b59f0 con 0x7f1f000b0ea0 2022-11-27T16:46:49.025 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1f071a1700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f000b6590 con 0x7f1f000b1ca0 2022-11-27T16:46:49.025 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1efffff700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1f000b0ea0 0x7f1f000ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.025 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1efffff700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f1f000b0ea0 0x7f1f000ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54850/0 (socket says 172.21.15.145:54850) 2022-11-27T16:46:49.026 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1efffff700 1 -- 172.21.15.145:0/1490254407 learned_addr learned my addr 172.21.15.145:0/1490254407 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:49.026 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1f0573e700 1 --2- 172.21.15.145:0/1490254407 >> v2:172.21.15.201:3300/0 conn(0x7f1f000b1ca0 0x7f1f000b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.026 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1efffff700 1 -- 172.21.15.145:0/1490254407 >> v2:172.21.15.201:3300/0 conn(0x7f1f000b1ca0 msgr2=0x7f1f000b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:49.026 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1efffff700 1 --2- 172.21.15.145:0/1490254407 >> v2:172.21.15.201:3300/0 conn(0x7f1f000b1ca0 0x7f1f000b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.027 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.014+0000 7f1efffff700 1 -- 172.21.15.145:0/1490254407 >> v1:172.21.15.201:6789/0 conn(0x7f1f000b0ad0 legacy=0x7f1f000b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.027 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1efffff700 1 -- 172.21.15.145:0/1490254407 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000cec90 con 0x7f1f000b0ea0 2022-11-27T16:46:49.027 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1efffff700 1 --2- 172.21.15.145:0/1490254407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b0ea0 0x7f1f000ce590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f0006d290 tx=0x7f1eec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1b3f23a730be66 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.027 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1eff7fe700 1 -- 172.21.15.145:0/1490254407 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011830 con 0x7f1f000b0ea0 2022-11-27T16:46:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1eff7fe700 1 -- 172.21.15.145:0/1490254407 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec011990 con 0x7f1f000b0ea0 2022-11-27T16:46:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1eff7fe700 1 -- 172.21.15.145:0/1490254407 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011d20 con 0x7f1f000b0ea0 2022-11-27T16:46:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 -- 172.21.15.145:0/1490254407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b0ea0 msgr2=0x7f1f000ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.028 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 --2- 172.21.15.145:0/1490254407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b0ea0 0x7f1f000ce590 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1f0006d290 tx=0x7f1eec009bd0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 -- 172.21.15.145:0/1490254407 shutdown_connections 2022-11-27T16:46:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 --2- 172.21.15.145:0/1490254407 >> v2:172.21.15.201:3300/0 conn(0x7f1f000b1ca0 0x7f1f000b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.029 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 --2- 172.21.15.145:0/1490254407 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b0ea0 0x7f1f000ce590 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.030 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 -- 172.21.15.145:0/1490254407 >> 172.21.15.145:0/1490254407 conn(0x7f1f000999c0 msgr2=0x7f1f000d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:49.030 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 -- 172.21.15.145:0/1490254407 shutdown_connections 2022-11-27T16:46:49.030 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.015+0000 7f1f071a1700 1 -- 172.21.15.145:0/1490254407 wait complete. 2022-11-27T16:46:49.034 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 Processor -- start 2022-11-27T16:46:49.034 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 -- start start 2022-11-27T16:46:49.034 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f1f000b0ad0 0x7f1f0015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.035 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f1f000b0ea0 0x7f1f000c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.035 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b1ca0 0x7f1f001eebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.035 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0015c490 con 0x7f1f000b1ca0 2022-11-27T16:46:49.035 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f001f2b50 con 0x7f1f000b0ea0 2022-11-27T16:46:49.036 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.017+0000 7f1f071a1700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0015b910 con 0x7f1f000b0ad0 2022-11-27T16:46:49.036 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b1ca0 0x7f1f001eebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.036 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b1ca0 0x7f1f001eebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54856/0 (socket says 172.21.15.145:54856) 2022-11-27T16:46:49.036 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 -- 172.21.15.145:0/186803628 learned_addr learned my addr 172.21.15.145:0/186803628 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:49.037 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1efffff700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f1f000b0ea0 0x7f1f000c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.037 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f04f3d700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f1f000b0ad0 0x7f1f0015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.037 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 -- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f1f000b0ad0 msgr2=0x7f1f0015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.037 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f1f000b0ad0 0x7f1f0015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.037 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 -- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f1f000b0ea0 msgr2=0x7f1f000c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.038 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f1f000b0ea0 0x7f1f000c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.038 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 -- 172.21.15.145:0/186803628 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec009550 con 0x7f1f000b1ca0 2022-11-27T16:46:49.038 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f0573e700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b1ca0 0x7f1f001eebf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800efc0 tx=0x7f1ef800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.038 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1efd7fa700 1 -- 172.21.15.145:0/186803628 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8009ea0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.039 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1efd7fa700 1 -- 172.21.15.145:0/186803628 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800f040 con 0x7f1f000b1ca0 2022-11-27T16:46:49.039 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0015bbf0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.039 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1efd7fa700 1 -- 172.21.15.145:0/186803628 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef800ced0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.039 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.018+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0015c1b0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.040 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.019+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f000739b0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.040 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.020+0000 7f1efd7fa700 1 -- 172.21.15.145:0/186803628 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f1ef8014480 con 0x7f1f000b1ca0 2022-11-27T16:46:49.040 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.020+0000 7f1efd7fa700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f1ee8066590 0x7f1ee8068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.040 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.020+0000 7f1efd7fa700 1 -- 172.21.15.145:0/186803628 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f1ef8083df0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.040 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.020+0000 7f1f04f3d700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f1ee8066590 0x7f1ee8068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.041 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.020+0000 7f1f04f3d700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f1ee8066590 0x7f1ee8068a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4007fd0 tx=0x7f1ef4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.041 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.024+0000 7f1efd7fa700 1 -- 172.21.15.145:0/186803628 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f1ef80517f0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.206 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.204+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f000b5680 con 0x7f1f000b1ca0 2022-11-27T16:46:49.207 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.205+0000 7f1efd7fa700 1 -- 172.21.15.145:0/186803628 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ef80180a0 con 0x7f1f000b1ca0 2022-11-27T16:46:49.207 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 1 2022-11-27T16:46:49.208 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f1ee8066590 msgr2=0x7f1ee8068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.209 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f1ee8066590 0x7f1ee8068a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4007fd0 tx=0x7f1ef4009040 comp rx=0 tx=0).stop 2022-11-27T16:46:49.209 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b1ca0 msgr2=0x7f1f001eebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.209 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b1ca0 0x7f1f001eebf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1ef800efc0 tx=0x7f1ef800c3a0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.209 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 shutdown_connections 2022-11-27T16:46:49.210 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f1f000b1ca0 0x7f1f001eebf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.210 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f1ee8066590 0x7f1ee8068a40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.210 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f1f000b0ad0 0x7f1f0015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.211 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 --2- 172.21.15.145:0/186803628 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f1f000b0ea0 0x7f1f000c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.211 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 >> 172.21.15.145:0/186803628 conn(0x7f1f000999c0 msgr2=0x7f1f00159c00 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:49.211 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 shutdown_connections 2022-11-27T16:46:49.212 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.207+0000 7f1f071a1700 1 -- 172.21.15.145:0/186803628 wait complete. 2022-11-27T16:46:49.221 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2022-11-27T16:46:49.344 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.342+0000 7fd496937700 1 Processor -- start 2022-11-27T16:46:49.344 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.342+0000 7fd496937700 1 -- start start 2022-11-27T16:46:49.348 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd496937700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900cbc30 0x7fd490159db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.348 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd496937700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7fd4900708a0 0x7fd490070d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.348 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd496937700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49015a2f0 con 0x7fd4900ae7e0 2022-11-27T16:46:49.348 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd496937700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49015d7b0 con 0x7fd4900708a0 2022-11-27T16:46:49.349 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd496937700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49015c8c0 con 0x7fd4900cbc30 2022-11-27T16:46:49.349 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd48f7fe700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fd4900ae7e0 0x7fd4900cc000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49736/0 (socket says 172.21.15.145:49736) 2022-11-27T16:46:49.349 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd48f7fe700 1 -- 172.21.15.145:0/2001684860 learned_addr learned my addr 172.21.15.145:0/2001684860 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:49.349 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd48ffff700 1 --2- 172.21.15.145:0/2001684860 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900cbc30 0x7fd490159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.350 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd494ed4700 1 --2- 172.21.15.145:0/2001684860 >> v2:172.21.15.201:3300/0 conn(0x7fd4900708a0 0x7fd490070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.350 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd48effd700 1 -- 172.21.15.145:0/2001684860 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746175977 0 0) 0x7fd480001110 con 0x7fd4900ae7e0 2022-11-27T16:46:49.350 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.343+0000 7fd48effd700 1 -- 172.21.15.145:0/2001684860 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd47c0035b0 con 0x7fd4900ae7e0 2022-11-27T16:46:49.351 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48effd700 1 -- 172.21.15.145:0/2001684860 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1458425454 0 0) 0x7fd47c0035b0 con 0x7fd4900ae7e0 2022-11-27T16:46:49.351 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48ffff700 1 -- 172.21.15.145:0/2001684860 >> v2:172.21.15.201:3300/0 conn(0x7fd4900708a0 msgr2=0x7fd490070d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:49.351 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48ffff700 1 --2- 172.21.15.145:0/2001684860 >> v2:172.21.15.201:3300/0 conn(0x7fd4900708a0 0x7fd490070d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.352 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48ffff700 1 -- 172.21.15.145:0/2001684860 >> v1:172.21.15.145:6789/0 conn(0x7fd4900ae7e0 legacy=0x7fd4900cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.352 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48ffff700 1 -- 172.21.15.145:0/2001684860 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49015cba0 con 0x7fd4900cbc30 2022-11-27T16:46:49.353 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48ffff700 1 --2- 172.21.15.145:0/2001684860 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900cbc30 0x7fd490159db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd484009820 tx=0x7fd484009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8aeaa3823a3d1c server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.353 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48effd700 1 -- 172.21.15.145:0/2001684860 <== mon.2 v2:172.21.15.201:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd480001800 con 0x7fd4900cbc30 2022-11-27T16:46:49.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48effd700 1 -- 172.21.15.145:0/2001684860 <== mon.2 v2:172.21.15.201:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd484005c30 con 0x7fd4900cbc30 2022-11-27T16:46:49.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd48effd700 1 -- 172.21.15.145:0/2001684860 <== mon.2 v2:172.21.15.201:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd484004030 con 0x7fd4900cbc30 2022-11-27T16:46:49.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd496937700 1 -- 172.21.15.145:0/2001684860 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900cbc30 msgr2=0x7fd490159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.354 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd496937700 1 --2- 172.21.15.145:0/2001684860 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900cbc30 0x7fd490159db0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd484009820 tx=0x7fd484009b30 comp rx=0 tx=0).stop 2022-11-27T16:46:49.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd496937700 1 -- 172.21.15.145:0/2001684860 shutdown_connections 2022-11-27T16:46:49.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd496937700 1 --2- 172.21.15.145:0/2001684860 >> v2:172.21.15.201:3300/0 conn(0x7fd4900708a0 0x7fd490070d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.356 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.344+0000 7fd496937700 1 --2- 172.21.15.145:0/2001684860 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900cbc30 0x7fd490159db0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.356 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.345+0000 7fd496937700 1 -- 172.21.15.145:0/2001684860 >> 172.21.15.145:0/2001684860 conn(0x7fd49006a490 msgr2=0x7fd49006a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:49.356 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.345+0000 7fd496937700 1 -- 172.21.15.145:0/2001684860 shutdown_connections 2022-11-27T16:46:49.357 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.345+0000 7fd496937700 1 -- 172.21.15.145:0/2001684860 wait complete. 2022-11-27T16:46:49.357 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.346+0000 7fd496937700 1 Processor -- start 2022-11-27T16:46:49.357 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.346+0000 7fd496937700 1 -- start start 2022-11-27T16:46:49.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900708a0 0x7fd4900ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd4900ae7e0 0x7fd490150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4900cbc30 0x7fd4900ac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490150f80 con 0x7fd4900cbc30 2022-11-27T16:46:49.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900ad5f0 con 0x7fd4900ae7e0 2022-11-27T16:46:49.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4900ad8d0 con 0x7fd4900708a0 2022-11-27T16:46:49.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4900cbc30 0x7fd4900ac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4900cbc30 0x7fd4900ac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54866/0 (socket says 172.21.15.145:54866) 2022-11-27T16:46:49.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 -- 172.21.15.145:0/657997617 learned_addr learned my addr 172.21.15.145:0/657997617 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:49.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd48f7fe700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd4900ae7e0 0x7fd490150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd48ffff700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900708a0 0x7fd4900ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 -- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900708a0 msgr2=0x7fd4900ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900708a0 0x7fd4900ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 -- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd4900ae7e0 msgr2=0x7fd490150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd4900ae7e0 0x7fd490150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 -- 172.21.15.145:0/657997617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484009430 con 0x7fd4900cbc30 2022-11-27T16:46:49.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd494ed4700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4900cbc30 0x7fd4900ac680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd48800efc0 tx=0x7fd48800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd48cff9700 1 -- 172.21.15.145:0/657997617 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488009ea0 con 0x7fd4900cbc30 2022-11-27T16:46:49.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd48cff9700 1 -- 172.21.15.145:0/657997617 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800f040 con 0x7fd4900cbc30 2022-11-27T16:46:49.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4900adbb0 con 0x7fd4900cbc30 2022-11-27T16:46:49.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd48cff9700 1 -- 172.21.15.145:0/657997617 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800ced0 con 0x7fd4900cbc30 2022-11-27T16:46:49.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.347+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4900ae170 con 0x7fd4900cbc30 2022-11-27T16:46:49.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.348+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4900ae390 con 0x7fd4900cbc30 2022-11-27T16:46:49.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.349+0000 7fd48cff9700 1 -- 172.21.15.145:0/657997617 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fd488014480 con 0x7fd4900cbc30 2022-11-27T16:46:49.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.349+0000 7fd48cff9700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd4700665c0 0x7fd470068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.349+0000 7fd48cff9700 1 -- 172.21.15.145:0/657997617 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fd488083d60 con 0x7fd4900cbc30 2022-11-27T16:46:49.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.349+0000 7fd48ffff700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd4700665c0 0x7fd470068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.350+0000 7fd48ffff700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd4700665c0 0x7fd470068a70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd484000c00 tx=0x7fd484008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.353+0000 7fd48cff9700 1 -- 172.21.15.145:0/657997617 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fd488051760 con 0x7fd4900cbc30 2022-11-27T16:46:49.533 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.531+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd4900739b0 con 0x7fd4900cbc30 2022-11-27T16:46:49.533 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.532+0000 7fd48cff9700 1 -- 172.21.15.145:0/657997617 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd488018020 con 0x7fd4900cbc30 2022-11-27T16:46:49.534 INFO:teuthology.orchestra.run.smithi145.stdout:pg_num: 8 2022-11-27T16:46:49.536 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd4700665c0 msgr2=0x7fd470068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.536 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd4700665c0 0x7fd470068a70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd484000c00 tx=0x7fd484008070 comp rx=0 tx=0).stop 2022-11-27T16:46:49.536 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4900cbc30 msgr2=0x7fd4900ac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.536 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4900cbc30 0x7fd4900ac680 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd48800efc0 tx=0x7fd48800c3a0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.537 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 shutdown_connections 2022-11-27T16:46:49.537 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd4900cbc30 0x7fd4900ac680 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.537 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd4700665c0 0x7fd470068a70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.538 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd4900708a0 0x7fd4900ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.538 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 --2- 172.21.15.145:0/657997617 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd4900ae7e0 0x7fd490150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.538 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 >> 172.21.15.145:0/657997617 conn(0x7fd49006a490 msgr2=0x7fd490070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:49.538 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 shutdown_connections 2022-11-27T16:46:49.538 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.534+0000 7fd496937700 1 -- 172.21.15.145:0/657997617 wait complete. 2022-11-27T16:46:49.548 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2022-11-27T16:46:49.668 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.666+0000 7f442bd8b700 1 Processor -- start 2022-11-27T16:46:49.668 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.666+0000 7f442bd8b700 1 -- start start 2022-11-27T16:46:49.679 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f442bd8b700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7f442415a0d0 0x7f44240af610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.680 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f442bd8b700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240b0550 con 0x7f44240aece0 2022-11-27T16:46:49.680 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f442bd8b700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240af130 con 0x7f44240ce9a0 2022-11-27T16:46:49.681 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f442bd8b700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44240af3a0 con 0x7f442415a0d0 2022-11-27T16:46:49.681 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f4429326700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f44240aece0 0x7f44240ce290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49746/0 (socket says 172.21.15.145:49746) 2022-11-27T16:46:49.682 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f4429326700 1 -- 172.21.15.145:0/2924551104 learned_addr learned my addr 172.21.15.145:0/2924551104 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:49.682 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f4429b27700 1 --2- 172.21.15.145:0/2924551104 >> v2:172.21.15.201:3300/0 conn(0x7f442415a0d0 0x7f44240af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.682 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f4428b25700 1 -- 172.21.15.145:0/2924551104 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651119633 0 0) 0x7f44240b0550 con 0x7f44240aece0 2022-11-27T16:46:49.683 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.667+0000 7f4428b25700 1 -- 172.21.15.145:0/2924551104 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f440c0035b0 con 0x7f44240aece0 2022-11-27T16:46:49.683 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4428b25700 1 -- 172.21.15.145:0/2924551104 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2657338543 0 0) 0x7f440c0035b0 con 0x7f44240aece0 2022-11-27T16:46:49.684 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4429b27700 1 -- 172.21.15.145:0/2924551104 >> v1:172.21.15.201:6789/0 conn(0x7f44240ce9a0 legacy=0x7f44240b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.684 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4429b27700 1 -- 172.21.15.145:0/2924551104 >> v1:172.21.15.145:6789/0 conn(0x7f44240aece0 legacy=0x7f44240ce290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.685 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4429b27700 1 -- 172.21.15.145:0/2924551104 --> v2:172.21.15.201:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240afc20 con 0x7f442415a0d0 2022-11-27T16:46:49.685 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4429b27700 1 --2- 172.21.15.145:0/2924551104 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f442415a0d0 0x7f44240af610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4414009820 tx=0x7f4414009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79beeb1be3b9384 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.686 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4428b25700 1 -- 172.21.15.145:0/2924551104 <== mon.1 v2:172.21.15.201:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420001670 con 0x7f442415a0d0 2022-11-27T16:46:49.686 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4428b25700 1 -- 172.21.15.145:0/2924551104 <== mon.1 v2:172.21.15.201:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414005c30 con 0x7f442415a0d0 2022-11-27T16:46:49.687 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.668+0000 7f4428b25700 1 -- 172.21.15.145:0/2924551104 <== mon.1 v2:172.21.15.201:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414004030 con 0x7f442415a0d0 2022-11-27T16:46:49.687 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.669+0000 7f442bd8b700 1 -- 172.21.15.145:0/2924551104 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f442415a0d0 msgr2=0x7f44240af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.688 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.669+0000 7f442bd8b700 1 --2- 172.21.15.145:0/2924551104 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f442415a0d0 0x7f44240af610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4414009820 tx=0x7f4414009b30 comp rx=0 tx=0).stop 2022-11-27T16:46:49.688 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.669+0000 7f442bd8b700 1 -- 172.21.15.145:0/2924551104 shutdown_connections 2022-11-27T16:46:49.689 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.669+0000 7f442bd8b700 1 --2- 172.21.15.145:0/2924551104 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f442415a0d0 0x7f44240af610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.689 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.669+0000 7f442bd8b700 1 -- 172.21.15.145:0/2924551104 >> 172.21.15.145:0/2924551104 conn(0x7f4424099a00 msgr2=0x7f44240d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:49.689 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.669+0000 7f442bd8b700 1 -- 172.21.15.145:0/2924551104 shutdown_connections 2022-11-27T16:46:49.690 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.669+0000 7f442bd8b700 1 -- 172.21.15.145:0/2924551104 wait complete. 2022-11-27T16:46:49.690 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.670+0000 7f442bd8b700 1 Processor -- start 2022-11-27T16:46:49.691 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.670+0000 7f442bd8b700 1 -- start start 2022-11-27T16:46:49.691 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f44240aece0 0x7f442414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.692 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44240ce9a0 0x7f44240ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.692 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f442415a0d0 0x7f442406a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.693 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44241504b0 con 0x7f44240ce9a0 2022-11-27T16:46:49.693 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44240cbba0 con 0x7f44240aece0 2022-11-27T16:46:49.694 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44240cbe80 con 0x7f442415a0d0 2022-11-27T16:46:49.694 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44240ce9a0 0x7f44240ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.695 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44240ce9a0 0x7f44240ccf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54876/0 (socket says 172.21.15.145:54876) 2022-11-27T16:46:49.695 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 -- 172.21.15.145:0/1364265477 learned_addr learned my addr 172.21.15.145:0/1364265477 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:49.696 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429b27700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f44240aece0 0x7f442414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.696 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442a328700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f442415a0d0 0x7f442406a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 -- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f442415a0d0 msgr2=0x7f442406a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.697 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f442415a0d0 0x7f442406a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.698 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 -- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f44240aece0 msgr2=0x7f442414ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.698 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f44240aece0 0x7f442414ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.699 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 -- 172.21.15.145:0/1364265477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009430 con 0x7f44240ce9a0 2022-11-27T16:46:49.699 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f4429326700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44240ce9a0 0x7f44240ccf20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f442000c760 tx=0x7f442000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.700 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f44127fc700 1 -- 172.21.15.145:0/1364265477 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420007560 con 0x7f44240ce9a0 2022-11-27T16:46:49.700 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f44127fc700 1 -- 172.21.15.145:0/1364265477 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44200076c0 con 0x7f44240ce9a0 2022-11-27T16:46:49.701 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f44127fc700 1 -- 172.21.15.145:0/1364265477 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420004ab0 con 0x7f44240ce9a0 2022-11-27T16:46:49.701 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44240cc160 con 0x7f44240ce9a0 2022-11-27T16:46:49.702 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.671+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44240cc720 con 0x7f44240ce9a0 2022-11-27T16:46:49.702 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.672+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44240739b0 con 0x7f44240ce9a0 2022-11-27T16:46:49.703 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.673+0000 7f44127fc700 1 -- 172.21.15.145:0/1364265477 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f442001a020 con 0x7f44240ce9a0 2022-11-27T16:46:49.703 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.673+0000 7f44127fc700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f4404066590 0x7f4404068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.704 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.673+0000 7f44127fc700 1 -- 172.21.15.145:0/1364265477 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f44200836d0 con 0x7f44240ce9a0 2022-11-27T16:46:49.704 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.673+0000 7f4429b27700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f4404066590 0x7f4404068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:49.705 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.674+0000 7f4429b27700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f4404066590 0x7f4404068a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4414000c00 tx=0x7f4414008070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:49.705 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.677+0000 7f44127fc700 1 -- 172.21.15.145:0/1364265477 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f44200511d0 con 0x7f44240ce9a0 2022-11-27T16:46:49.858 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.856+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f44240cfbb0 con 0x7f44240ce9a0 2022-11-27T16:46:49.859 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.856+0000 7f44127fc700 1 -- 172.21.15.145:0/1364265477 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f44200547f0 con 0x7f44240ce9a0 2022-11-27T16:46:49.859 INFO:teuthology.orchestra.run.smithi145.stdout: 2022-11-27T16:46:49.860 INFO:teuthology.orchestra.run.smithi145.stdout:[ 2022-11-27T16:46:49.860 INFO:teuthology.orchestra.run.smithi145.stdout: { 2022-11-27T16:46:49.860 INFO:teuthology.orchestra.run.smithi145.stdout: "poolnum": 1, 2022-11-27T16:46:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: "poolname": ".mgr" 2022-11-27T16:46:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: }, 2022-11-27T16:46:49.861 INFO:teuthology.orchestra.run.smithi145.stdout: { 2022-11-27T16:46:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: "poolnum": 2, 2022-11-27T16:46:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: "poolname": "rbd" 2022-11-27T16:46:49.862 INFO:teuthology.orchestra.run.smithi145.stdout: } 2022-11-27T16:46:49.863 INFO:teuthology.orchestra.run.smithi145.stdout:] 2022-11-27T16:46:49.865 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f4404066590 msgr2=0x7f4404068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.865 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f4404066590 0x7f4404068a40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4414000c00 tx=0x7f4414008070 comp rx=0 tx=0).stop 2022-11-27T16:46:49.866 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44240ce9a0 msgr2=0x7f44240ccf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:49.868 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44240ce9a0 0x7f44240ccf20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f442000c760 tx=0x7f442000ca70 comp rx=0 tx=0).stop 2022-11-27T16:46:49.868 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 shutdown_connections 2022-11-27T16:46:49.868 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f4404066590 0x7f4404068a40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.869 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f442415a0d0 0x7f442406a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.869 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f44240aece0 0x7f442414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.869 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 --2- 172.21.15.145:0/1364265477 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f44240ce9a0 0x7f44240ccf20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:49.869 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 >> 172.21.15.145:0/1364265477 conn(0x7f4424099a00 msgr2=0x7f44240b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:49.870 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 shutdown_connections 2022-11-27T16:46:49.870 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.858+0000 7f442bd8b700 1 -- 172.21.15.145:0/1364265477 wait complete. 2022-11-27T16:46:49.872 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2022-11-27T16:46:49.873 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2022-11-27T16:46:49.996 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.994+0000 7fdd0fe94700 1 Processor -- start 2022-11-27T16:46:49.997 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0fe94700 1 -- start start 2022-11-27T16:46:49.999 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0fe94700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdd080cad50 0x7fdd080ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:49.999 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0fe94700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cead0 0x7fdd0806a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:50.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0fe94700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0806a4e0 con 0x7fdd0815c570 2022-11-27T16:46:50.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0fe94700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd080cddb0 con 0x7fdd080cad50 2022-11-27T16:46:50.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0fe94700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd080b2030 con 0x7fdd080cead0 2022-11-27T16:46:50.000 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0d42f700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdd080cad50 0x7fdd080ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:50.001 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.995+0000 7fdd0d42f700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7fdd080cad50 0x7fdd080ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54882/0 (socket says 172.21.15.145:54882) 2022-11-27T16:46:50.001 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0d42f700 1 -- 172.21.15.145:0/3692530771 learned_addr learned my addr 172.21.15.145:0/3692530771 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:50.001 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0e431700 1 --2- 172.21.15.145:0/3692530771 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cead0 0x7fdd0806a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:50.001 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0d42f700 1 -- 172.21.15.145:0/3692530771 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cead0 msgr2=0x7fdd0806a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:50.002 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0d42f700 1 --2- 172.21.15.145:0/3692530771 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cead0 0x7fdd0806a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:50.002 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0d42f700 1 -- 172.21.15.145:0/3692530771 >> v1:172.21.15.201:6789/0 conn(0x7fdd0815c570 legacy=0x7fdd080cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:50.003 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0d42f700 1 -- 172.21.15.145:0/3692530771 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd080cc500 con 0x7fdd080cad50 2022-11-27T16:46:50.003 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0d42f700 1 --2- 172.21.15.145:0/3692530771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cad50 0x7fdd080ce590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd040098f0 tx=0x7fdd04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cee05a389c4180 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:50.004 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0cc2e700 1 -- 172.21.15.145:0/3692530771 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400b8a0 con 0x7fdd080cad50 2022-11-27T16:46:50.004 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0cc2e700 1 -- 172.21.15.145:0/3692530771 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0400ba00 con 0x7fdd080cad50 2022-11-27T16:46:50.004 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.996+0000 7fdd0cc2e700 1 -- 172.21.15.145:0/3692530771 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0400bd90 con 0x7fdd080cad50 2022-11-27T16:46:50.005 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 -- 172.21.15.145:0/3692530771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cad50 msgr2=0x7fdd080ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:50.005 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/3692530771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cad50 0x7fdd080ce590 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdd040098f0 tx=0x7fdd04009c00 comp rx=0 tx=0).stop 2022-11-27T16:46:50.005 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 -- 172.21.15.145:0/3692530771 shutdown_connections 2022-11-27T16:46:50.005 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/3692530771 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cead0 0x7fdd0806a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:50.005 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/3692530771 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cad50 0x7fdd080ce590 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:50.006 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 -- 172.21.15.145:0/3692530771 >> 172.21.15.145:0/3692530771 conn(0x7fdd080999c0 msgr2=0x7fdd080d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:50.006 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 -- 172.21.15.145:0/3692530771 shutdown_connections 2022-11-27T16:46:50.006 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.997+0000 7fdd0fe94700 1 -- 172.21.15.145:0/3692530771 wait complete. 2022-11-27T16:46:50.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.998+0000 7fdd0fe94700 1 Processor -- start 2022-11-27T16:46:50.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.998+0000 7fdd0fe94700 1 -- start start 2022-11-27T16:46:50.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cad50 0x7fdd08158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:50.010 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cead0 0x7fdd0815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:50.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fdd0815c570 0x7fdd08159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:50.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd081599a0 con 0x7fdd080cead0 2022-11-27T16:46:50.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08158a70 con 0x7fdd0815c570 2022-11-27T16:46:50.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd08158d20 con 0x7fdd080cad50 2022-11-27T16:46:50.011 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cead0 0x7fdd0815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:50.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cead0 0x7fdd0815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54888/0 (socket says 172.21.15.145:54888) 2022-11-27T16:46:50.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 -- 172.21.15.145:0/2849271737 learned_addr learned my addr 172.21.15.145:0/2849271737 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:50.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0dc30700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cad50 0x7fdd08158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:50.012 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0e431700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fdd0815c570 0x7fdd08159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:50.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 -- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cad50 msgr2=0x7fdd08158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:50.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cad50 0x7fdd08158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:50.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 -- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fdd0815c570 msgr2=0x7fdd08159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:50.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fdd0815c570 0x7fdd08159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:50.013 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 -- 172.21.15.145:0/2849271737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04009550 con 0x7fdd080cead0 2022-11-27T16:46:50.014 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0d42f700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cead0 0x7fdd0815cc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd04000c00 tx=0x7fdd04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:50.014 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdcf67fc700 1 -- 172.21.15.145:0/2849271737 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd04004380 con 0x7fdd080cead0 2022-11-27T16:46:50.014 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdcf67fc700 1 -- 172.21.15.145:0/2849271737 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0401f070 con 0x7fdd080cead0 2022-11-27T16:46:50.014 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd08159000 con 0x7fdd080cead0 2022-11-27T16:46:50.015 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdcf67fc700 1 -- 172.21.15.145:0/2849271737 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd040036a0 con 0x7fdd080cead0 2022-11-27T16:46:50.015 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:49.999+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd081595c0 con 0x7fdd080cead0 2022-11-27T16:46:50.015 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.000+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd08159ae0 con 0x7fdd080cead0 2022-11-27T16:46:50.015 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.001+0000 7fdcf67fc700 1 -- 172.21.15.145:0/2849271737 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fdd0402a020 con 0x7fdd080cead0 2022-11-27T16:46:50.015 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.001+0000 7fdcf67fc700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fdce8066590 0x7fdce8068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:50.016 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.001+0000 7fdcf67fc700 1 -- 172.21.15.145:0/2849271737 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fdd04084ae0 con 0x7fdd080cead0 2022-11-27T16:46:50.016 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.001+0000 7fdd0dc30700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fdce8066590 0x7fdce8068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:50.016 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.001+0000 7fdd0dc30700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fdce8066590 0x7fdce8068a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007fd0 tx=0x7fdcf8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:50.016 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.005+0000 7fdcf67fc700 1 -- 172.21.15.145:0/2849271737 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fdd08159ae0 con 0x7fdd080cead0 2022-11-27T16:46:50.186 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:50.184+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fdd08159ae0 con 0x7fdd080cead0 2022-11-27T16:46:50.671 DEBUG:teuthology.orchestra.run.smithi145:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:46:50.674 DEBUG:teuthology.orchestra.run.smithi201:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2022-11-27T16:46:51.189 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.187+0000 7fdcf67fc700 1 -- 172.21.15.145:0/2849271737 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7fdd08159ae0 con 0x7fdd080cead0 2022-11-27T16:46:51.193 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fdce8066590 msgr2=0x7fdce8068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:51.193 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fdce8066590 0x7fdce8068a40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8007fd0 tx=0x7fdcf8009040 comp rx=0 tx=0).stop 2022-11-27T16:46:51.194 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cead0 msgr2=0x7fdd0815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:51.194 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cead0 0x7fdd0815cc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd04000c00 tx=0x7fdd04004750 comp rx=0 tx=0).stop 2022-11-27T16:46:51.195 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 shutdown_connections 2022-11-27T16:46:51.195 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fdd0815c570 0x7fdd08159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.195 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fdce8066590 0x7fdce8068a40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.196 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fdd080cad50 0x7fdd08158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.196 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 --2- 172.21.15.145:0/2849271737 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fdd080cead0 0x7fdd0815cc60 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd04000c00 tx=0x7fdd04004750 comp rx=0 tx=0).stop 2022-11-27T16:46:51.196 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 >> 172.21.15.145:0/2849271737 conn(0x7fdd080999c0 msgr2=0x7fdd0806bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:51.197 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 shutdown_connections 2022-11-27T16:46:51.197 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.191+0000 7fdd0fe94700 1 -- 172.21.15.145:0/2849271737 wait complete. 2022-11-27T16:46:51.198 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_metadata' created 2022-11-27T16:46:51.208 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2022-11-27T16:46:51.341 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.340+0000 7fd6f0bd8700 1 Processor -- start 2022-11-27T16:46:51.342 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.340+0000 7fd6f0bd8700 1 -- start start 2022-11-27T16:46:51.360 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6f0bd8700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 0x7fd6ec1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.360 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6f0bd8700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7fd6ec069380 0x7fd6ec069790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.360 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6f0bd8700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec069cd0 con 0x7fd6ec0cd0d0 2022-11-27T16:46:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6f0bd8700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec15d300 con 0x7fd6ec069380 2022-11-27T16:46:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6f0bd8700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b1f90 con 0x7fd6ec156120 2022-11-27T16:46:51.361 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e9d9b700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fd6ec0cd0d0 0x7fd6ec071380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49770/0 (socket says 172.21.15.145:49770) 2022-11-27T16:46:51.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e9d9b700 1 -- 172.21.15.145:0/1327315902 learned_addr learned my addr 172.21.15.145:0/1327315902 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:51.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6ead9d700 1 --2- 172.21.15.145:0/1327315902 >> v2:172.21.15.201:3300/0 conn(0x7fd6ec069380 0x7fd6ec069790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6ea59c700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 0x7fd6ec1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742956544 0 0) 0x7fd6ec069cd0 con 0x7fd6ec0cd0d0 2022-11-27T16:46:51.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d00035b0 con 0x7fd6ec0cd0d0 2022-11-27T16:46:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3866326092 0 0) 0x7fd6d00035b0 con 0x7fd6ec0cd0d0 2022-11-27T16:46:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 msgr2=0x7fd6ec1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:51.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 0x7fd6ec1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 >> v2:172.21.15.201:3300/0 conn(0x7fd6ec069380 msgr2=0x7fd6ec069790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 --2- 172.21.15.145:0/1327315902 >> v2:172.21.15.201:3300/0 conn(0x7fd6ec069380 0x7fd6ec069790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec156a30 con 0x7fd6ec0cd0d0 2022-11-27T16:46:51.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1749869138 0 0) 0x7fd6e0001670 con 0x7fd6ec0cd0d0 2022-11-27T16:46:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 >> v1:172.21.15.145:6789/0 conn(0x7fd6ec0cd0d0 legacy=0x7fd6ec071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 0x7fd6d0003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6d0003f80 0x7fd6d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6d0004d90 0x7fd6d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec15d300 con 0x7fd6d0004d90 2022-11-27T16:46:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec0b1f90 con 0x7fd6d0003f80 2022-11-27T16:46:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.341+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d000ba10 con 0x7fd6ec156120 2022-11-27T16:46:51.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ead9d700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6d0004d90 0x7fd6d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ea59c700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 0x7fd6d0003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6e9d9b700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6d0003f80 0x7fd6d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ead9d700 1 -- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 msgr2=0x7fd6d0003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:51.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ead9d700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6ec156120 0x7fd6d0003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.368 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ead9d700 1 -- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6d0003f80 msgr2=0x7fd6d0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:51.368 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ead9d700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6d0003f80 0x7fd6d0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.368 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ead9d700 1 -- 172.21.15.145:0/1327315902 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00035b0 con 0x7fd6d0004d90 2022-11-27T16:46:51.368 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6ead9d700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6d0004d90 0x7fd6d0005200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fc0 tx=0x7fd6d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f38dfe4c1045fe1 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:51.368 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e400ec30 con 0x7fd6d0004d90 2022-11-27T16:46:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e400f040 con 0x7fd6d0004d90 2022-11-27T16:46:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.342+0000 7fd6e959a700 1 -- 172.21.15.145:0/1327315902 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4014480 con 0x7fd6d0004d90 2022-11-27T16:46:51.369 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.343+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/1327315902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6d0004d90 msgr2=0x7fd6d0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.343+0000 7fd6f0bd8700 1 --2- 172.21.15.145:0/1327315902 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6d0004d90 0x7fd6d0005200 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009fc0 tx=0x7fd6d0005880 comp rx=0 tx=0).stop 2022-11-27T16:46:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.343+0000 7fd6ead9d700 1 -- 172.21.15.145:0/1327315902 reap_dead start 2022-11-27T16:46:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.343+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/1327315902 shutdown_connections 2022-11-27T16:46:51.370 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.343+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/1327315902 >> 172.21.15.145:0/1327315902 conn(0x7fd6ec06a490 msgr2=0x7fd6ec06a860 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:51.371 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.343+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/1327315902 shutdown_connections 2022-11-27T16:46:51.371 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.343+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/1327315902 wait complete. 2022-11-27T16:46:51.371 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.344+0000 7fd6f0bd8700 1 Processor -- start 2022-11-27T16:46:51.371 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.344+0000 7fd6f0bd8700 1 -- start start 2022-11-27T16:46:51.371 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6f0bd8700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6d0004d90 0x7fd6ec154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6f0bd8700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6ec15b890 0x7fd6ec158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6f0bd8700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6ec15bd00 0x7fd6ec158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6f0bd8700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec15b400 con 0x7fd6ec15b890 2022-11-27T16:46:51.372 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6f0bd8700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1552c0 con 0x7fd6ec15bd00 2022-11-27T16:46:51.373 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6f0bd8700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec15a7f0 con 0x7fd6d0004d90 2022-11-27T16:46:51.373 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6ec15b890 0x7fd6ec158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.373 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6ec15b890 0x7fd6ec158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54908/0 (socket says 172.21.15.145:54908) 2022-11-27T16:46:51.373 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 -- 172.21.15.145:0/904824892 learned_addr learned my addr 172.21.15.145:0/904824892 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:51.373 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6ea59c700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6d0004d90 0x7fd6ec154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.374 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6ead9d700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6ec15bd00 0x7fd6ec158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.374 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 -- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6d0004d90 msgr2=0x7fd6ec154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:51.374 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6d0004d90 0x7fd6ec154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.374 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 -- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6ec15bd00 msgr2=0x7fd6ec158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:51.375 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6ec15bd00 0x7fd6ec158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:51.375 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 -- 172.21.15.145:0/904824892 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec15aad0 con 0x7fd6ec15b890 2022-11-27T16:46:51.375 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6ead9d700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6ec15bd00 0x7fd6ec158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:51.375 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.345+0000 7fd6e9d9b700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6ec15b890 0x7fd6ec158950 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0007310 tx=0x7fd6e0007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:51.376 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.346+0000 7fd6daffd700 1 -- 172.21.15.145:0/904824892 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000f6c0 con 0x7fd6ec15b890 2022-11-27T16:46:51.376 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.346+0000 7fd6daffd700 1 -- 172.21.15.145:0/904824892 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e000f820 con 0x7fd6ec15b890 2022-11-27T16:46:51.376 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.346+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec15adb0 con 0x7fd6ec15b890 2022-11-27T16:46:51.376 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.346+0000 7fd6daffd700 1 -- 172.21.15.145:0/904824892 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000fbb0 con 0x7fd6ec15b890 2022-11-27T16:46:51.376 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.346+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec1583e0 con 0x7fd6ec15b890 2022-11-27T16:46:51.377 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.347+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec0cdfd0 con 0x7fd6ec15b890 2022-11-27T16:46:51.377 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.347+0000 7fd6daffd700 1 -- 172.21.15.145:0/904824892 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7fd6e000fd10 con 0x7fd6ec15b890 2022-11-27T16:46:51.377 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.347+0000 7fd6daffd700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6c80665c0 0x7fd6c8068a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:51.377 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.347+0000 7fd6daffd700 1 -- 172.21.15.145:0/904824892 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7fd6e0083700 con 0x7fd6ec15b890 2022-11-27T16:46:51.377 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.347+0000 7fd6ea59c700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6c80665c0 0x7fd6c8068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:51.378 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.348+0000 7fd6ea59c700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6c80665c0 0x7fd6c8068a70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009c40 tx=0x7fd6dc0091c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:51.378 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.352+0000 7fd6daffd700 1 -- 172.21.15.145:0/904824892 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7fd6e0050ff0 con 0x7fd6ec15b890 2022-11-27T16:46:51.533 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:51.532+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fd6ec0739b0 con 0x7fd6ec15b890 2022-11-27T16:46:52.197 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.196+0000 7fd6daffd700 1 -- 172.21.15.145:0/904824892 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7fd6e0054610 con 0x7fd6ec15b890 2022-11-27T16:46:52.208 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6c80665c0 msgr2=0x7fd6c8068a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.209 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6c80665c0 0x7fd6c8068a70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009c40 tx=0x7fd6dc0091c0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.209 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6ec15b890 msgr2=0x7fd6ec158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.209 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6ec15b890 0x7fd6ec158950 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0007310 tx=0x7fd6e0007620 comp rx=0 tx=0).stop 2022-11-27T16:46:52.210 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 shutdown_connections 2022-11-27T16:46:52.210 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fd6ec15bd00 0x7fd6ec158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.210 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7fd6c80665c0 0x7fd6c8068a70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.211 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fd6d0004d90 0x7fd6ec154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.211 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 --2- 172.21.15.145:0/904824892 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fd6ec15b890 0x7fd6ec158950 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.211 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.207+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 >> 172.21.15.145:0/904824892 conn(0x7fd6ec06a490 msgr2=0x7fd6ec0b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:52.212 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.208+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 shutdown_connections 2022-11-27T16:46:52.212 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.209+0000 7fd6f0bd8700 1 -- 172.21.15.145:0/904824892 wait complete. 2022-11-27T16:46:52.213 INFO:teuthology.orchestra.run.smithi145.stderr:pool 'cephfs_data' created 2022-11-27T16:46:52.226 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2022-11-27T16:46:52.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.353+0000 7f59b5d5a700 1 Processor -- start 2022-11-27T16:46:52.355 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.353+0000 7f59b5d5a700 1 -- start start 2022-11-27T16:46:52.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59b5d5a700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.362 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59b5d5a700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f59b00b1ca0 0x7f59b00b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59b5d5a700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b00d1570 con 0x7f59b00b0ad0 2022-11-27T16:46:52.363 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59b5d5a700 1 -- --> v2:172.21.15.145:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b00b59f0 con 0x7f59b00b1ca0 2022-11-27T16:46:52.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59b5d5a700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b00b6590 con 0x7f59b00b0ea0 2022-11-27T16:46:52.364 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59affff700 1 --2- >> v2:172.21.15.145:3300/0 conn(0x7f59b00b1ca0 0x7f59b00b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59af7fe700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7f59b00b0ad0 0x7f59b00b5e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49790/0 (socket says 172.21.15.145:49790) 2022-11-27T16:46:52.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59af7fe700 1 -- 172.21.15.145:0/2259223380 learned_addr learned my addr 172.21.15.145:0/2259223380 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:52.365 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59aeffd700 1 --2- 172.21.15.145:0/2259223380 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.366 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59affff700 1 -- 172.21.15.145:0/2259223380 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 msgr2=0x7f59b00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:52.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59affff700 1 --2- 172.21.15.145:0/2259223380 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b00ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.367 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59affff700 1 -- 172.21.15.145:0/2259223380 >> v1:172.21.15.145:6789/0 conn(0x7f59b00b0ad0 legacy=0x7f59b00b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59affff700 1 -- 172.21.15.145:0/2259223380 --> v2:172.21.15.145:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b00cec90 con 0x7f59b00b1ca0 2022-11-27T16:46:52.368 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59ae7fc700 1 -- 172.21.15.145:0/2259223380 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 175354961 0 0) 0x7f59b00d1570 con 0x7f59b00b0ad0 2022-11-27T16:46:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.354+0000 7f59aeffd700 1 --2- 172.21.15.145:0/2259223380 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2022-11-27T16:46:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59affff700 1 --2- 172.21.15.145:0/2259223380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b1ca0 0x7f59b00b44f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009d90 tx=0x7f59a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298b2f28bb5e091e server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:52.369 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59ae7fc700 1 -- 172.21.15.145:0/2259223380 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a800e070 con 0x7f59b00b1ca0 2022-11-27T16:46:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59ae7fc700 1 -- 172.21.15.145:0/2259223380 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a800bbd0 con 0x7f59b00b1ca0 2022-11-27T16:46:52.370 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59ae7fc700 1 -- 172.21.15.145:0/2259223380 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8012430 con 0x7f59b00b1ca0 2022-11-27T16:46:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 -- 172.21.15.145:0/2259223380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b1ca0 msgr2=0x7f59b00b44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.371 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/2259223380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b1ca0 0x7f59b00b44f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009d90 tx=0x7f59a800b330 comp rx=0 tx=0).stop 2022-11-27T16:46:52.372 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 -- 172.21.15.145:0/2259223380 shutdown_connections 2022-11-27T16:46:52.372 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/2259223380 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b1ca0 0x7f59b00b44f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.373 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/2259223380 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.373 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 -- 172.21.15.145:0/2259223380 >> 172.21.15.145:0/2259223380 conn(0x7f59b00999c0 msgr2=0x7f59b00d0690 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:52.374 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 -- 172.21.15.145:0/2259223380 shutdown_connections 2022-11-27T16:46:52.374 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.355+0000 7f59b5d5a700 1 -- 172.21.15.145:0/2259223380 wait complete. 2022-11-27T16:46:52.374 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.356+0000 7f59b5d5a700 1 Processor -- start 2022-11-27T16:46:52.375 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.357+0000 7f59b5d5a700 1 -- start start 2022-11-27T16:46:52.375 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.357+0000 7f59b5d5a700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b0ad0 0x7f59b0150f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.376 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.357+0000 7f59b5d5a700 1 --2- >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.376 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.357+0000 7f59b5d5a700 1 --2- >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f59b00b1ca0 0x7f59b015d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.357+0000 7f59b5d5a700 1 -- --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b00ae8d0 con 0x7f59b00b0ad0 2022-11-27T16:46:52.377 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.357+0000 7f59b5d5a700 1 -- --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b00aebb0 con 0x7f59b00b1ca0 2022-11-27T16:46:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.357+0000 7f59b5d5a700 1 -- --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b00aee90 con 0x7f59b00b0ea0 2022-11-27T16:46:52.378 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b0ad0 0x7f59b0150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.379 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 --2- >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b0ad0 0x7f59b0150f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.145:3300/0 says I am v2:172.21.15.145:54920/0 (socket says 172.21.15.145:54920) 2022-11-27T16:46:52.379 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 -- 172.21.15.145:0/3571222309 learned_addr learned my addr 172.21.15.145:0/3571222309 (peer_addr_for_me v2:172.21.15.145:0/0) 2022-11-27T16:46:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59aeffd700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.380 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59affff700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f59b00b1ca0 0x7f59b015d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.381 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 -- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 msgr2=0x7f59b015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.381 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b015cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.381 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 -- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f59b00b1ca0 msgr2=0x7f59b015d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.382 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f59b00b1ca0 0x7f59b015d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.382 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 -- 172.21.15.145:0/3571222309 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8009a70 con 0x7f59b00b0ad0 2022-11-27T16:46:52.383 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59affff700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f59b00b1ca0 0x7f59b015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2022-11-27T16:46:52.383 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59af7fe700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b0ad0 0x7f59b0150f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f599800c550 tx=0x7f599800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:52.384 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59a7fff700 1 -- 172.21.15.145:0/3571222309 <== mon.0 v2:172.21.15.145:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998004c80 con 0x7f59b00b0ad0 2022-11-27T16:46:52.384 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59a7fff700 1 -- 172.21.15.145:0/3571222309 <== mon.0 v2:172.21.15.145:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998004e00 con 0x7f59b00b0ad0 2022-11-27T16:46:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b00af170 con 0x7f59b00b0ad0 2022-11-27T16:46:52.385 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59a7fff700 1 -- 172.21.15.145:0/3571222309 <== mon.0 v2:172.21.15.145:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998007310 con 0x7f59b00b0ad0 2022-11-27T16:46:52.386 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.358+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b00ae380 con 0x7f59b00b0ad0 2022-11-27T16:46:52.386 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.359+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b00739b0 con 0x7f59b00b0ad0 2022-11-27T16:46:52.386 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.359+0000 7f59a7fff700 1 -- 172.21.15.145:0/3571222309 <== mon.0 v2:172.21.15.145:3300/0 4 ==== mgrmap(e 4) v1 ==== 84497+0+0 (secure 0 0 0) 0x7f599801a020 con 0x7f59b00b0ad0 2022-11-27T16:46:52.387 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.360+0000 7f59a7fff700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f5994066590 0x7f5994068a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.387 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.360+0000 7f59a7fff700 1 -- 172.21.15.145:0/3571222309 <== mon.0 v2:172.21.15.145:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5958+0+0 (secure 0 0 0) 0x7f59980832e0 con 0x7f59b00b0ad0 2022-11-27T16:46:52.388 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.360+0000 7f59aeffd700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f5994066590 0x7f5994068a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.388 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.360+0000 7f59aeffd700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f5994066590 0x7f5994068a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59b0154aa0 tx=0x7f59a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2022-11-27T16:46:52.390 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.364+0000 7f59a7fff700 1 -- 172.21.15.145:0/3571222309 <== mon.0 v2:172.21.15.145:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+170936 (secure 0 0 0) 0x7f59980509f0 con 0x7f59b00b0ad0 2022-11-27T16:46:52.588 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.585+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f59b00ae600 con 0x7f59b00b0ad0 2022-11-27T16:46:52.595 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.592+0000 7f59a7fff700 1 -- 172.21.15.145:0/3571222309 <== mon.0 v2:172.21.15.145:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5998054010 con 0x7f59b00b0ad0 2022-11-27T16:46:52.602 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f5994066590 msgr2=0x7f5994068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.603 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f5994066590 0x7f5994068a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f59b0154aa0 tx=0x7f59a0008040 comp rx=0 tx=0).stop 2022-11-27T16:46:52.603 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b0ad0 msgr2=0x7f59b0150f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.603 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b0ad0 0x7f59b0150f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f599800c550 tx=0x7f599800c860 comp rx=0 tx=0).stop 2022-11-27T16:46:52.603 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 shutdown_connections 2022-11-27T16:46:52.604 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7f59b00b1ca0 0x7f59b015d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.604 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7f59b00b0ad0 0x7f59b0150f50 secure :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f599800c550 tx=0x7f599800c860 comp rx=0 tx=0).stop 2022-11-27T16:46:52.604 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.145:6832/96576,v1:172.21.15.145:6833/96576] conn(0x7f5994066590 0x7f5994068a40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.604 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 --2- 172.21.15.145:0/3571222309 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7f59b00b0ea0 0x7f59b015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.599+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 >> 172.21.15.145:0/3571222309 conn(0x7f59b00999c0 msgr2=0x7f59b0159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2022-11-27T16:46:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.600+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 shutdown_connections 2022-11-27T16:46:52.605 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.600+0000 7f59b5d5a700 1 -- 172.21.15.145:0/3571222309 wait complete. 2022-11-27T16:46:52.606 INFO:teuthology.orchestra.run.smithi145.stderr:new fs with metadata pool 3 and data pool 4 2022-11-27T16:46:52.621 DEBUG:teuthology.orchestra.run.smithi145:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2022-11-27T16:46:52.740 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.739+0000 7fcdce8f3700 1 Processor -- start 2022-11-27T16:46:52.741 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.739+0000 7fcdce8f3700 1 -- start start 2022-11-27T16:46:52.748 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.739+0000 7fcdce8f3700 1 --2- >> v2:172.21.15.201:3300/0 conn(0x7fcdc80adec0 0x7fcdc8160640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.748 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.739+0000 7fcdce8f3700 1 -- --> v1:172.21.15.145:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc80cc220 con 0x7fcdc806c580 2022-11-27T16:46:52.749 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdce8f3700 1 -- --> v1:172.21.15.201:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc815fb60 con 0x7fcdc806a5a0 2022-11-27T16:46:52.749 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdce8f3700 1 -- --> v2:172.21.15.201:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc815fe40 con 0x7fcdc80adec0 2022-11-27T16:46:52.749 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdcce90700 1 --1- >> v1:172.21.15.145:6789/0 conn(0x7fcdc806c580 0x7fcdc806fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.145:6789/0 says I am v1:172.21.15.145:49816/0 (socket says 172.21.15.145:49816) 2022-11-27T16:46:52.750 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdcce90700 1 -- 172.21.15.145:0/2349539347 learned_addr learned my addr 172.21.15.145:0/2349539347 (peer_addr_for_me v1:172.21.15.145:0/0) 2022-11-27T16:46:52.750 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 <== mon.0 v1:172.21.15.145:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1826672209 0 0) 0x7fcdc80cc220 con 0x7fcdc806c580 2022-11-27T16:46:52.750 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 --> v1:172.21.15.145:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb40035b0 con 0x7fcdc806c580 2022-11-27T16:46:52.750 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc7fff700 1 --2- 172.21.15.145:0/2349539347 >> v2:172.21.15.201:3300/0 conn(0x7fcdc80adec0 0x7fcdc8160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.751 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 <== mon.0 v1:172.21.15.145:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2053753841 0 0) 0x7fcdb40035b0 con 0x7fcdc806c580 2022-11-27T16:46:52.751 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 >> v2:172.21.15.201:3300/0 conn(0x7fcdc80adec0 msgr2=0x7fcdc8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:52.751 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 --2- 172.21.15.145:0/2349539347 >> v2:172.21.15.201:3300/0 conn(0x7fcdc80adec0 0x7fcdc8160640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.751 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 >> v1:172.21.15.201:6789/0 conn(0x7fcdc806a5a0 legacy=0x7fcdc806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.751 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 --> v1:172.21.15.145:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc8160120 con 0x7fcdc806c580 2022-11-27T16:46:52.752 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 <== mon.0 v1:172.21.15.145:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1749869138 0 0) 0x7fcdc00019f0 con 0x7fcdc806c580 2022-11-27T16:46:52.752 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 >> v1:172.21.15.145:6789/0 conn(0x7fcdc806c580 legacy=0x7fcdc806fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2022-11-27T16:46:52.752 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdb4003a00 0x7fcdb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.752 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fcdb4004850 0x7fcdb4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.753 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fcdb4005200 0x7fcdb400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2022-11-27T16:46:52.753 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 --> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc815fe40 con 0x7fcdb4003a00 2022-11-27T16:46:52.753 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 --> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb400c080 con 0x7fcdb4004850 2022-11-27T16:46:52.753 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc6ffd700 1 -- 172.21.15.145:0/2349539347 --> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb400c240 con 0x7fcdb4005200 2022-11-27T16:46:52.753 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc7fff700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.145:3300/0,v1:172.21.15.145:6789/0] conn(0x7fcdb4003a00 0x7fcdb4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.754 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdc77fe700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fcdb4004850 0x7fcdb4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.754 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.740+0000 7fcdcce90700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fcdb4005200 0x7fcdb400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2022-11-27T16:46:52.754 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.741+0000 7fcdc7fff700 1 -- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fcdb4005200 msgr2=0x7fcdb400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:52.754 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.741+0000 7fcdc7fff700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3301/0,v1:172.21.15.201:6790/0] conn(0x7fcdb4005200 0x7fcdb400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.754 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.741+0000 7fcdc7fff700 1 -- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fcdb4004850 msgr2=0x7fcdb4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2022-11-27T16:46:52.755 INFO:teuthology.orchestra.run.smithi145.stderr:2022-11-27T16:46:52.741+0000 7fcdc7fff700 1 --2- 172.21.15.145:0/2349539347 >> [v2:172.21.15.201:3300/0,v1:172.21.15.201:6789/0] conn(0x7fcdb4004850 0x7fcdb4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2022-11-27T16:46:52.755 INFO:teuthology.orchestra.run.smith